Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [............X..........X.XX......XX.XX.XXXXX.X..............] duration: 5.832865521s FAIL (2025-07-29 18:18:56 UTC) orbiter warning: uplink: context deadline exceeded Duration: 5.832890278s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 1.850179291s FAIL (2025-07-29 18:18:44 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.850206011s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.....................................XXXXXXXXXXX............] duration: 5.880848044s FAIL (2025-07-29 18:18:42 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.880875236s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [......................................XXXXXXXXXXX...........] duration: 5.891365842s FAIL (2025-07-29 18:18:03 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.891388626s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 1.898279721s FAIL (2025-07-29 18:18:03 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.898322865s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [.............X..........X.XX......XX.XX.XXXXX.X.............] duration: 5.720494696s FAIL (2025-07-29 18:17:57 UTC) orbiter warning: uplink: context deadline exceeded Duration: 5.720518002s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 1.831818867s FAIL (2025-07-29 18:17:48 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.831842773s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.......................................XXXXXXXXXXX..........] duration: 5.816893624s FAIL (2025-07-29 18:17:28 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.816915437s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 1.82782052s FAIL (2025-07-29 18:17:13 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.827842762s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [..............X..........X.XX......XX.XX.XXXXX.X............] duration: 5.915904419s FAIL (2025-07-29 18:17:05 UTC) orbiter warning: uplink: context deadline exceeded Duration: 5.915932754s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 1.88452286s FAIL (2025-07-29 18:16:38 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.8845487s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [........................................XXXXXXXXXXX.........] duration: 5.670252797s FAIL (2025-07-29 18:16:31 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.670270232s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 1.887339578s FAIL (2025-07-29 18:16:17 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.887370898s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXX........] duration: 5.73847981s FAIL (2025-07-29 18:16:14 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.73850064s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [...............X..........X.XX......XX.XX.XXXXX.X...........] duration: 5.57143589s FAIL (2025-07-29 18:16:14 UTC) orbiter warning: uplink: context deadline exceeded Duration: 5.571464614s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.958912468s FAIL (2025-07-29 18:15:41 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.95895525s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [................X..........X.XX......XX.XX.XXXXX.X..........] duration: 5.561897369s FAIL (2025-07-29 18:15:26 UTC) orbiter warning: uplink: context deadline exceeded Duration: 5.561927947s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 1.883339425s FAIL (2025-07-29 18:15:18 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.883371486s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXX.......] duration: 5.804023917s FAIL (2025-07-29 18:15:13 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.804055798s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [.................X..........X.XX......XX.XX.XXXXX.X.........] duration: 5.651329992s FAIL (2025-07-29 18:15:04 UTC) orbiter warning: uplink: context deadline exceeded Duration: 5.651361342s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.905750022s FAIL (2025-07-29 18:14:33 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.905780289s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...........................................XXXXXXXXXXX......] duration: 5.759568028s FAIL (2025-07-29 18:14:18 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.759594188s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [..................X..........X.XX......XX.XX.XXXXX.X........] duration: 5.730247971s FAIL (2025-07-29 18:14:00 UTC) orbiter warning: uplink: context deadline exceeded Duration: 5.730270835s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.988560838s FAIL (2025-07-29 18:13:50 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.988586066s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [...................X..........X.XX......XX.XX.XXXXX.X.......] duration: 5.739995057s FAIL (2025-07-29 18:13:39 UTC) orbiter warning: uplink: context deadline exceeded Duration: 5.740023623s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [............................................XXXXXXXXXXX.....] duration: 5.836234014s FAIL (2025-07-29 18:13:33 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.836264391s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [....................X..........X.XX......XX.XX.XXXXX.X......] duration: 5.919616455s FAIL (2025-07-29 18:12:43 UTC) orbiter warning: uplink: context deadline exceeded Duration: 5.919638267s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 1.888882538s FAIL (2025-07-29 18:12:37 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.888911303s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.............................................XXXXXXXXXXX....] duration: 6.076591209s FAIL (2025-07-29 18:12:22 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 6.076615916s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [.....................X..........X.XX......XX.XX.XXXXX.X.....] duration: 5.921379241s FAIL (2025-07-29 18:11:59 UTC) orbiter warning: uplink: context deadline exceeded Duration: 5.921408496s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.220542315s FAIL (2025-07-29 18:11:49 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 2.220570269s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [......................X..........X.XX......XX.XX.XXXXX.X....] duration: 6.004494313s FAIL (2025-07-29 18:11:33 UTC) orbiter warning: uplink: context deadline exceeded Duration: 6.004557574s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..............................................XXXXXXXXXXX...] duration: 5.776395407s FAIL (2025-07-29 18:11:30 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.776424223s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [.......................X..........X.XX......XX.XX.XXXXX.X...] duration: 5.748889016s FAIL (2025-07-29 18:11:01 UTC) orbiter warning: uplink: context deadline exceeded Duration: 5.748920805s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXX..] duration: 5.693694522s FAIL (2025-07-29 18:10:54 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.693721894s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [........................X..........X.XX......XX.XX.XXXXX.X..] duration: 6.069732226s FAIL (2025-07-29 18:10:38 UTC) orbiter warning: uplink: context deadline exceeded Duration: 6.069780708s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X.................................X.........................] duration: 2.046161058s FAIL (2025-07-29 18:10:37 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 2.046195604s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X.................................X........................] duration: 1.894969317s FAIL (2025-07-29 18:10:15 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.895016547s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXX.] duration: 22.527538198s FAIL (2025-07-29 18:10:08 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 22.527558496s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [.........................X..........X.XX......XX.XX.XXXXX.X.] duration: 8.505100826s FAIL (2025-07-29 18:10:05 UTC) orbiter warning: uplink: context deadline exceeded Duration: 8.505126185s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.................................................XXXXXXXXXXX] duration: 30.002275971s FAIL (2025-07-29 18:09:42 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 30.002286531s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [..........................X..........X.XX......XX.XX.XXXXX.X] duration: 30.002414817s FAIL (2025-07-29 18:09:40 UTC) orbiter warning: uplink: context deadline exceeded Duration: 30.002425036s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..................................................XXXXXXXXXX] duration: 30.001948841s FAIL (2025-07-29 18:09:12 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== LI5DITGX5A, seg_pos={0 0}, upload=24272]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Jul 29 18:07:38.131675025 (25h21m50.60761349s) [seg_pos={0 0}, upload=24272]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:07:38.131764316 (25h21m50.607702791s) [upload=24272]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:07:38.131772401 (25h21m50.607710936s) [upload=24272]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:07:38.527924890 (25h21m51.003863355s) [upload=24272]: Done uploading segments... Jul 29 18:07:38.527942053 (25h21m51.003880518s) [upload=24272]: Upload finished. err: <nil> Jul 29 18:08:43.176330729 (25h22m55.652269194s) [upload=24289]: Starting upload Jul 29 18:08:43.176411805 (25h22m55.65235028s) [upload=24289]: Done starting upload Jul 29 18:08:43.176489834 (25h22m55.652428299s) [upload=24289]: Uploading segments... Jul 29 18:08:43.176707140 (25h22m55.652645605s) [upload=24289]: Got next segment. Inline: false Jul 29 18:08:43.176756394 (25h22m55.652694859s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:08:43.176762135 (25h22m55.6527006s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:08:43.412967323 (25h22m55.888905798s) [seg_pos={0 0}, upload=24289]: Begin upload segment... Jul 29 18:08:43.413017619 (25h22m55.888956084s) [seg_pos={0 0}, upload=24289]: Done begin upload segment. Jul 29 18:08:43.413043158 (25h22m55.888981623s) [noise=true , address=104.36.82.2:4730 , piece_id=USQQXU2LRC56BCFHP5SHIATJAR6MVDCKJZP3GIQI3SLXC2SKSBZQ, seg_pos={0 0}, upload=24289]: Uploading piece... Jul 29 18:08:43.413113212 (25h22m55.889051687s) [noise=true , address=45.91.116.5:4297 , piece_id=OK4LQC7ZJRU6F6VEIFLQFJ4QHUZDQ3Y4T5FKYLHS7BWI6EKR6OOA, seg_pos={0 0}, upload=24289]: Uploading piece... Jul 29 18:08:43.413774778 (25h22m55.889713243s) [noise=true , address=45.91.116.6:4973 , piece_id=UZIPYP3CVPLQ73CPG5F6XZTZHUY2SD332TSJH6IRUG76OOMVEK2Q, seg_pos={0 0}, upload=24289]: Uploading piece... Jul 29 18:08:43.414067358 (25h22m55.890005833s) [noise=true , address=77.90.31.12:4592 , piece_id=CJFCNV6ZMTT74GE73YOCCQPHSTIKUBC34JTOORMR7M43WM7VAMSQ, seg_pos={0 0}, upload=24289]: Uploading piece... Jul 29 18:08:43.414384334 (25h22m55.890322799s) [noise=true , address=45.91.116.6:4974 , piece_id=7WLWKTMV7EKOQRTWTMCC7KEBGTSWSM4KU2CTNCVSA4IRRTMES72Q, seg_pos={0 0}, upload=24289]: Uploading piece... Jul 29 18:08:43.414799439 (25h22m55.890737914s) [noise=true , address=104.36.82.4:4802 , piece_id=O7KV4GKVLMBLQ6HWGD3PQ7F4QLY7W4SI2O3XOKG35XSFR6DMX3LA, seg_pos={0 0}, upload=24289]: Uploading piece... Jul 29 18:08:43.414831941 (25h22m55.890770406s) [noise=true , address=77.90.31.18:4806 , piece_id=S445CBWVJFOKCKKDHG2OGRYV3HLX6CHOVEOWCUAJ4MCE4GRUDC2A, seg_pos={0 0}, upload=24289]: Uploading piece... Jul 29 18:08:43.415348690 (25h22m55.891287155s) [noise=true , address=45.91.116.14:4191 , piece_id=PDBJX5Y6CW6YDGABSBFRRLI6M4NARW6OFV735MTWCIEKBO33AHQA, seg_pos={0 0}, upload=24289]: Uploading piece... Jul 29 18:08:43.415459582 (25h22m55.891398037s) [noise=true , address=77.90.31.15:4697 , piece_id=2LPJKMPGCJFOL3HU3GEE5BLVHGMOJFUWQZSAXSG4XH6PRJZQHVXA, seg_pos={0 0}, upload=24289]: Uploading piece... Jul 29 18:08:43.415751620 (25h22m55.891690085s) [noise=true , address=45.91.116.7:4009 , piece_id=I4CDOU74CTLWYF2KJM6YTRTFFPHPOP4EHGEK6OXKOEYNTKB2CHNQ, seg_pos={0 0}, upload=24289]: Uploading piece... Jul 29 18:08:43.416152437 (25h22m55.892090902s) [noise=true , address=45.91.116.9:4262 , piece_id=FYDK3HYJURXOC3ZH43WL6M6JMLZ52MUY5G5LZIYSA4ZNIJEWUQGA, seg_pos={0 0}, upload=24289]: Uploading piece... Jul 29 18:08:43.416417304 (25h22m55.892355779s) [noise=true , address=77.90.31.13:4628 , piece_id=GM72HJDK5MK4IWKYUF2WHUVV6RLUQXW5FK4OY4HH52RDHKQEXLJQ, seg_pos={0 0}, upload=24289]: Uploading piece... Jul 29 18:08:43.416655029 (25h22m55.892593494s) [noise=true , address=104.36.82.13:4185 , piece_id=ZMJFBGCMTG4RW7LFTR6TFJLEC6YGZ3YAOHQFJXOK33O2UCVFV6QA, seg_pos={0 0}, upload=24289]: Uploading piece... Jul 29 18:08:43.416980752 (25h22m55.892919217s) [noise=true , address=45.91.116.8:4047 , piece_id=MCV2DY4YXPKYDTVEALVBYTZILK4MXA23GJPSTTLJ2CFPFA7LPLRA, seg_pos={0 0}, upload=24289]: Uploading piece... Jul 29 18:08:43.417452715 (25h22m55.89339118s) [noise=true , address=77.90.31.21:4879 , piece_id=QARDUHDHU377NF5TF5FKKBVCUVCBKEOYV4LGF6KQWB6K6F7IDEGQ, seg_pos={0 0}, upload=24289]: Uploading piece... Jul 29 18:08:43.417851949 (25h22m55.893790414s) [noise=true , address=77.90.31.14:4662 , piece_id=522FHIDDDCZOXN7VWE45UXL6GH5WDQPYJR7HMIUT75PKHFQYMXGA, seg_pos={0 0}, upload=24289]: Uploading piece... Jul 29 18:08:45.344695291 (25h22m57.820633756s) [upload=24289]: Next returned nil segment Jul 29 18:08:45.344717584 (25h22m57.820656059s) [upload=24289]: Waiting for error group managing segments... Jul 29 18:08:45.861617763 (25h22m58.337556228s) [noise=true , address=77.90.31.15:4697 , piece_id=2LPJKMPGCJFOL3HU3GEE5BLVHGMOJFUWQZSAXSG4XH6PRJZQHVXA, seg_pos={0 0}, upload=24289]: Done uploading piece. err: <nil> Jul 29 18:08:45.864578157 (25h22m58.340516622s) [noise=true , address=45.91.116.6:4973 , piece_id=UZIPYP3CVPLQ73CPG5F6XZTZHUY2SD332TSJH6IRUG76OOMVEK2Q, seg_pos={0 0}, upload=24289]: Done uploading piece. err: <nil> Jul 29 18:08:45.865374040 (25h22m58.341312505s) [noise=true , address=77.90.31.14:4662 , piece_id=522FHIDDDCZOXN7VWE45UXL6GH5WDQPYJR7HMIUT75PKHFQYMXGA, seg_pos={0 0}, upload=24289]: Done uploading piece. err: <nil> Jul 29 18:08:45.867053393 (25h22m58.342991858s) [noise=true , address=104.36.82.4:4802 , piece_id=O7KV4GKVLMBLQ6HWGD3PQ7F4QLY7W4SI2O3XOKG35XSFR6DMX3LA, seg_pos={0 0}, upload=24289]: Done uploading piece. err: <nil> Jul 29 18:08:45.868739779 (25h22m58.344678244s) [noise=true , address=45.91.116.5:4297 , piece_id=OK4LQC7ZJRU6F6VEIFLQFJ4QHUZDQ3Y4T5FKYLHS7BWI6EKR6OOA, seg_pos={0 0}, upload=24289]: Done uploading piece. err: <nil> Jul 29 18:08:45.870315044 (25h22m58.346253509s) [noise=true , address=104.36.82.13:4185 , piece_id=ZMJFBGCMTG4RW7LFTR6TFJLEC6YGZ3YAOHQFJXOK33O2UCVFV6QA, seg_pos={0 0}, upload=24289]: Done uploading piece. err: <nil> Jul 29 18:08:45.873506260 (25h22m58.349444725s) [noise=true , address=45.91.116.9:4262 , piece_id=FYDK3HYJURXOC3ZH43WL6M6JMLZ52MUY5G5LZIYSA4ZNIJEWUQGA, seg_pos={0 0}, upload=24289]: Done uploading piece. err: <nil> Jul 29 18:08:45.883031053 (25h22m58.358969518s) [noise=true , address=77.90.31.21:4879 , piece_id=QARDUHDHU377NF5TF5FKKBVCUVCBKEOYV4LGF6KQWB6K6F7IDEGQ, seg_pos={0 0}, upload=24289]: Done uploading piece. err: <nil> Jul 29 18:08:45.885896446 (25h22m58.361834941s) [noise=true , address=77.90.31.13:4628 , piece_id=GM72HJDK5MK4IWKYUF2WHUVV6RLUQXW5FK4OY4HH52RDHKQEXLJQ, seg_pos={0 0}, upload=24289]: Done uploading piece. err: <nil> Jul 29 18:08:45.891393664 (25h22m58.367332129s) [noise=true , address=104.36.82.2:4730 , piece_id=USQQXU2LRC56BCFHP5SHIATJAR6MVDCKJZP3GIQI3SLXC2SKSBZQ, seg_pos={0 0}, upload=24289]: Done uploading piece. err: <nil> Jul 29 18:08:45.891799060 (25h22m58.367737525s) [noise=true , address=45.91.116.7:4009 , piece_id=I4CDOU74CTLWYF2KJM6YTRTFFPHPOP4EHGEK6OXKOEYNTKB2CHNQ, seg_pos={0 0}, upload=24289]: Done uploading piece. err: <nil> Jul 29 18:08:45.907850131 (25h22m58.383788596s) [noise=true , address=77.90.31.18:4806 , piece_id=S445CBWVJFOKCKKDHG2OGRYV3HLX6CHOVEOWCUAJ4MCE4GRUDC2A, seg_pos={0 0}, upload=24289]: Done uploading piece. err: <nil> Jul 29 18:08:45.931531210 (25h22m58.407469675s) [noise=true , address=45.91.116.6:4974 , piece_id=7WLWKTMV7EKOQRTWTMCC7KEBGTSWSM4KU2CTNCVSA4IRRTMES72Q, seg_pos={0 0}, upload=24289]: Done uploading piece. err: <nil> Jul 29 18:08:45.931549515 (25h22m58.40748798s) [seg_pos={0 0}, upload=24289]: Segment reached optimal threshold of 13 pieces. Jul 29 18:08:45.931656049 (25h22m58.407594514s) [noise=true , address=45.91.116.8:4047 , piece_id=MCV2DY4YXPKYDTVEALVBYTZILK4MXA23GJPSTTLJ2CFPFA7LPLRA, seg_pos={0 0}, upload=24289]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Jul 29 18:08:45.931865430 (25h22m58.407803895s) [noise=true , address=45.91.116.14:4191 , piece_id=PDBJX5Y6CW6YDGABSBFRRLI6M4NARW6OFV735MTWCIEKBO33AHQA, seg_pos={0 0}, upload=24289]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Jul 29 18:08:45.931899935 (25h22m58.4078384s) [noise=true , address=77.90.31.12:4592 , piece_id=CJFCNV6ZMTT74GE73YOCCQPHSTIKUBC34JTOORMR7M43WM7VAMSQ, seg_pos={0 0}, upload=24289]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Duration: 30.00196385s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 13 out of 60 is failed (threshold=3) [...........................X..........X.XX......XX.XX.XXXXX.] duration: 25.820185069s FAIL (2025-07-29 18:09:09 UTC) orbiter warning: uplink: context deadline exceeded Duration: 25.820228952s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X.................................X.......................] duration: 1.960507374s FAIL (2025-07-29 18:09:06 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.960530518s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X.................................X......................] duration: 2.056394526s FAIL (2025-07-29 18:08:44 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 2.05641751s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 13 out of 60 is failed (threshold=3) [............................X..........X.XX......XX.XX.XXXXX] duration: 30.002206798s FAIL (2025-07-29 18:08:07 UTC) orbiter warning: uplink: context deadline exceeded Duration: 30.002240493s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration: 30.00262489s FAIL (2025-07-29 18:08:03 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== HGQOX5IA, seg_pos={0 0}, upload=24257]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: write tcp 188.166.225.170:38698->77.90.31.17:4771: use of closed network connection; write tcp 188.166.225.170:38698->77.90.31.17:4771: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38698->77.90.31.17:4771: use of closed network connection Jul 29 18:06:39.897868413 (25h20m52.373806878s) [seg_pos={0 0}, upload=24257]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:06:39.897928167 (25h20m52.373866622s) [upload=24257]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:06:39.897932355 (25h20m52.37387082s) [upload=24257]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:06:40.217925898 (25h20m52.693864363s) [upload=24257]: Done uploading segments... Jul 29 18:06:40.217945646 (25h20m52.693884111s) [upload=24257]: Upload finished. err: <nil> Jul 29 18:07:34.344743998 (25h21m46.820682463s) [upload=24272]: Starting upload Jul 29 18:07:34.344806167 (25h21m46.820744632s) [upload=24272]: Done starting upload Jul 29 18:07:34.344910767 (25h21m46.820849232s) [upload=24272]: Uploading segments... Jul 29 18:07:34.345056375 (25h21m46.82099484s) [upload=24272]: Got next segment. Inline: false Jul 29 18:07:34.345105139 (25h21m46.821043604s) [upload=24272]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:07:34.345110519 (25h21m46.821048984s) [upload=24272]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:07:34.579006258 (25h21m47.054944733s) [seg_pos={0 0}, upload=24272]: Begin upload segment... Jul 29 18:07:34.579065792 (25h21m47.055004257s) [seg_pos={0 0}, upload=24272]: Done begin upload segment. Jul 29 18:07:34.579097052 (25h21m47.055035587s) [noise=true , address=104.36.82.9:4007 , piece_id=2LCDSDH3IUVYD5SJSLOTZQOLLNPSECMO3O4Q5OV3TGOIG7ZYD42A, seg_pos={0 0}, upload=24272]: Uploading piece... Jul 29 18:07:34.579113183 (25h21m47.055051648s) [noise=true , address=45.91.116.8:4045 , piece_id=LJTTWKYFASEMBFWJWHH7FUSDCWJUFI5U7KWRFV6BCQISN7LZPC5A, seg_pos={0 0}, upload=24272]: Uploading piece... Jul 29 18:07:34.579506736 (25h21m47.055445201s) [noise=true , address=45.91.116.4:4939 , piece_id=JAHLJHUORHUPZLG2LOGP4Z3OFG3I75PA2J2VKE5LDJLCCPRN3INQ, seg_pos={0 0}, upload=24272]: Uploading piece... Jul 29 18:07:34.579851314 (25h21m47.055789891s) [noise=true , address=104.36.82.3:4767 , piece_id=OMJAUHO2MAUNIU7TM4YGHOGMV2U46YHFHWAATDTRHEFWPTOQBQCA, seg_pos={0 0}, upload=24272]: Uploading piece... Jul 29 18:07:34.579883417 (25h21m47.055821892s) [noise=true , address=77.90.31.12:4592 , piece_id=4C2MDIUEEFR3PZZURNHK2O3COZ4VMA5U7C34RRFPEXSY3FYPC4YA, seg_pos={0 0}, upload=24272]: Uploading piece... Jul 29 18:07:34.580167289 (25h21m47.056105754s) [noise=true , address=45.91.116.12:4227 , piece_id=WOYY3Q65JFFK4J4EWK5SROCEG7TMTKXXBB2L3XISTDUBPAPWM2DA, seg_pos={0 0}, upload=24272]: Uploading piece... Jul 29 18:07:34.580496199 (25h21m47.056434654s) [noise=true , address=104.36.82.10:4078 , piece_id=4OXJDRMM2SOSXNJZQFMO7GBTK6JS5ALKRD26VNRHPRJE2JUZNBBA, seg_pos={0 0}, upload=24272]: Uploading piece... Jul 29 18:07:34.580565962 (25h21m47.056504427s) [noise=true , address=45.91.116.5:4299 , piece_id=TL4KTFH24S66P7FCY2OAOQYNEMM5ZMZ3AU7EO3BYZRSMLM6T2OEA, seg_pos={0 0}, upload=24272]: Uploading piece... Jul 29 18:07:34.580927134 (25h21m47.056865599s) [noise=true , address=77.90.31.14:4661 , piece_id=7RWX4FZLWOTXED3RDWV23GAZVVDMT7V6SU7DSSXWMPVTJRYJJCWQ, seg_pos={0 0}, upload=24272]: Uploading piece... Jul 29 18:07:34.580999061 (25h21m47.056937586s) [noise=true , address=77.90.31.15:4698 , piece_id=3Y2IN5BWWHQJ2GUAXOS6JJAQHDIPPAQETXM7JJQ5LUGO6BZQDNCA, seg_pos={0 0}, upload=24272]: Uploading piece... Jul 29 18:07:34.581153386 (25h21m47.057091851s) [noise=true , address=45.91.116.8:4048 , piece_id=Y37IUUSCI63OLTXTJSRM2LFKMOLNN6TO4OYFZWGJ3X3IYPWRIJOA, seg_pos={0 0}, upload=24272]: Uploading piece... Jul 29 18:07:34.581424705 (25h21m47.057363171s) [noise=true , address=45.91.116.6:4974 , piece_id=ZYWC63UUKBBIZWPXU4CAZZD743TTMIZ4LK6SJW44TXFRRR4PKY2A, seg_pos={0 0}, upload=24272]: Uploading piece... Jul 29 18:07:34.581484249 (25h21m47.057422714s) [noise=true , address=45.91.116.10:4082 , piece_id=DDHOMBSX46FZKVCUV7PKPGAXVP5OQP5NA34YFMWXGP2CXUNBKCEQ, seg_pos={0 0}, upload=24272]: Uploading piece... Jul 29 18:07:34.581985839 (25h21m47.057924304s) [noise=true , address=45.91.116.11:4118 , piece_id=TAEP2S56OREJQADGWXSNJAWIU6XPCFVPXF7C36JVW6ZD7HYHWJSA, seg_pos={0 0}, upload=24272]: Uploading piece... Jul 29 18:07:34.582348332 (25h21m47.058286797s) [noise=true , address=45.91.116.4:4938 , piece_id=IZ3HD3AUXK7QQ2T6DSPGFHO57J72WK33XCSM3Z7EJTLI5DITGX5A, seg_pos={0 0}, upload=24272]: Uploading piece... Jul 29 18:07:34.582738769 (25h21m47.058677234s) [noise=true , address=77.90.31.11:4555 , piece_id=UNCWE4CUFJCMORHVYATNLAH3BTBZK4CQWGYYRGJDZSKDJQF6XIJQ, seg_pos={0 0}, upload=24272]: Uploading piece... Jul 29 18:07:36.566279180 (25h21m49.042217645s) [upload=24272]: Next returned nil segment Jul 29 18:07:36.566303376 (25h21m49.042241841s) [upload=24272]: Waiting for error group managing segments... Jul 29 18:07:36.916306617 (25h21m49.392245092s) [noise=true , address=45.91.116.12:4227 , piece_id=WOYY3Q65JFFK4J4EWK5SROCEG7TMTKXXBB2L3XISTDUBPAPWM2DA, seg_pos={0 0}, upload=24272]: Done uploading piece. err: <nil> Jul 29 18:07:36.922841050 (25h21m49.398779515s) [noise=true , address=45.91.116.6:4974 , piece_id=ZYWC63UUKBBIZWPXU4CAZZD743TTMIZ4LK6SJW44TXFRRR4PKY2A, seg_pos={0 0}, upload=24272]: Done uploading piece. err: <nil> Jul 29 18:07:36.956263768 (25h21m49.432202233s) [noise=true , address=45.91.116.8:4048 , piece_id=Y37IUUSCI63OLTXTJSRM2LFKMOLNN6TO4OYFZWGJ3X3IYPWRIJOA, seg_pos={0 0}, upload=24272]: Done uploading piece. err: <nil> Jul 29 18:07:36.970650575 (25h21m49.44658904s) [noise=true , address=77.90.31.11:4555 , piece_id=UNCWE4CUFJCMORHVYATNLAH3BTBZK4CQWGYYRGJDZSKDJQF6XIJQ, seg_pos={0 0}, upload=24272]: Done uploading piece. err: <nil> Jul 29 18:07:36.981901101 (25h21m49.457839566s) [noise=true , address=45.91.116.10:4082 , piece_id=DDHOMBSX46FZKVCUV7PKPGAXVP5OQP5NA34YFMWXGP2CXUNBKCEQ, seg_pos={0 0}, upload=24272]: Done uploading piece. err: <nil> Jul 29 18:07:36.992615501 (25h21m49.468553966s) [noise=true , address=45.91.116.8:4045 , piece_id=LJTTWKYFASEMBFWJWHH7FUSDCWJUFI5U7KWRFV6BCQISN7LZPC5A, seg_pos={0 0}, upload=24272]: Done uploading piece. err: <nil> Jul 29 18:07:36.997786975 (25h21m49.47372544s) [noise=true , address=77.90.31.14:4661 , piece_id=7RWX4FZLWOTXED3RDWV23GAZVVDMT7V6SU7DSSXWMPVTJRYJJCWQ, seg_pos={0 0}, upload=24272]: Done uploading piece. err: <nil> Jul 29 18:07:37.009347394 (25h21m49.48528586s) [noise=true , address=77.90.31.15:4698 , piece_id=3Y2IN5BWWHQJ2GUAXOS6JJAQHDIPPAQETXM7JJQ5LUGO6BZQDNCA, seg_pos={0 0}, upload=24272]: Done uploading piece. err: <nil> Jul 29 18:07:37.027882629 (25h21m49.503821094s) [noise=true , address=104.36.82.3:4767 , piece_id=OMJAUHO2MAUNIU7TM4YGHOGMV2U46YHFHWAATDTRHEFWPTOQBQCA, seg_pos={0 0}, upload=24272]: Done uploading piece. err: <nil> Jul 29 18:07:37.033453617 (25h21m49.509392082s) [noise=true , address=77.90.31.12:4592 , piece_id=4C2MDIUEEFR3PZZURNHK2O3COZ4VMA5U7C34RRFPEXSY3FYPC4YA, seg_pos={0 0}, upload=24272]: Done uploading piece. err: <nil> Jul 29 18:07:37.404395492 (25h21m49.880333967s) [noise=true , address=104.36.82.9:4007 , piece_id=2LCDSDH3IUVYD5SJSLOTZQOLLNPSECMO3O4Q5OV3TGOIG7ZYD42A, seg_pos={0 0}, upload=24272]: Done uploading piece. err: <nil> Jul 29 18:07:37.407780949 (25h21m49.883719414s) [noise=true , address=45.91.116.11:4118 , piece_id=TAEP2S56OREJQADGWXSNJAWIU6XPCFVPXF7C36JVW6ZD7HYHWJSA, seg_pos={0 0}, upload=24272]: Done uploading piece. err: <nil> Jul 29 18:07:38.131413775 (25h21m50.60735223s) [noise=true , address=104.36.82.10:4078 , piece_id=4OXJDRMM2SOSXNJZQFMO7GBTK6JS5ALKRD26VNRHPRJE2JUZNBBA, seg_pos={0 0}, upload=24272]: Done uploading piece. err: <nil> Jul 29 18:07:38.131436769 (25h21m50.607375234s) [seg_pos={0 0}, upload=24272]: Segment reached optimal threshold of 13 pieces. Jul 29 18:07:38.131521371 (25h21m50.607459836s) [noise=true , address=45.91.116.4:4939 , piece_id=JAHLJHUORHUPZLG2LOGP4Z3OFG3I75PA2J2VKE5LDJLCCPRN3INQ, seg_pos={0 0}, upload=24272]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: expected piece hash; context canceled; EOF Jul 29 18:07:38.131578300 (25h21m50.607516755s) [noise=true , address=45.91.116.5:4299 , piece_id=TL4KTFH24S66P7FCY2OAOQYNEMM5ZMZ3AU7EO3BYZRSMLM6T2OEA, seg_pos={0 0}, upload=24272]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Jul 29 18:07:38.131638034 (25h21m50.607576499s) [noise=true , address=45.91.116.4:4938 , piece_id=IZ3HD3AUXK7QQ2T6DSPGFHO57J72WK33XCSM3Z7EJTLI5DITGX5A, seg_pos={0 0}, upload=24272]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Jul 29 18:07:38.131675025 (25h21m50.60761349s) [seg_pos={0 0}, upload=24272]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:07:38.131764316 (25h21m50.607702791s) [upload=24272]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:07:38.131772401 (25h21m50.607710936s) [upload=24272]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:07:38.527924890 (25h21m51.003863355s) [upload=24272]: Done uploading segments... Jul 29 18:07:38.527942053 (25h21m51.003880518s) [upload=24272]: Upload finished. err: <nil> Duration: 30.002653325s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [X...X.................................X.....................] duration: 1.85602361s FAIL (2025-07-29 18:07:34 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K Duration: 1.856053047s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 12 out of 60 is failed (threshold=3) [.............................X..........X.XX......XX.XX.XXXX] duration: 30.001624606s FAIL (2025-07-29 18:07:30 UTC) orbiter warning: uplink: context deadline exceeded Duration: 30.001641388s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)