Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [.....X.XX......XX.XX.XXXXX.X.................X..............] duration: 5.908320822s FAIL (2025-07-29 18:31:32 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M Duration: 5.90834521s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [....................XXXXXXXXXXX.............................] duration: 5.739316383s FAIL (2025-07-29 18:31:29 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.739343655s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [......X.XX......XX.XX.XXXXX.X.................X.............] duration: 6.639213836s FAIL (2025-07-29 18:31:10 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M Duration: 6.63923765s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [.......X.XX......XX.XX.XXXXX.X.................X............] duration: 5.955988451s FAIL (2025-07-29 18:30:41 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M Duration: 5.9560192s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.....................XXXXXXXXXXX............................] duration: 5.999700938s FAIL (2025-07-29 18:30:33 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.999726788s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......X.....................................X..............] duration: 2.040976071s FAIL (2025-07-29 18:30:33 UTC) orbiter warning: digest mismatch: after 961b4d588233cdffecfdee9a52135c19a25a5252ae31d78557f60dabdf9ef66b != before 15fa414527dc8e4383d128b95548ae532846d0ea58b26052bae13dc71f88b0e7 Duration: 2.041002652s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [........X.XX......XX.XX.XXXXX.X.................X...........] duration: 6.094828126s FAIL (2025-07-29 18:29:51 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M Duration: 6.094856029s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........X.....................................X.............] duration: 1.852300141s FAIL (2025-07-29 18:29:40 UTC) orbiter warning: digest mismatch: after 961b4d588233cdffecfdee9a52135c19a25a5252ae31d78557f60dabdf9ef66b != before 15fa414527dc8e4383d128b95548ae532846d0ea58b26052bae13dc71f88b0e7 Duration: 1.852322524s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [......................XXXXXXXXXXX...........................] duration: 5.829830137s FAIL (2025-07-29 18:29:25 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.82986301s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........X.....................................X............] duration: 1.852009233s FAIL (2025-07-29 18:28:56 UTC) orbiter warning: digest mismatch: after 961b4d588233cdffecfdee9a52135c19a25a5252ae31d78557f60dabdf9ef66b != before 15fa414527dc8e4383d128b95548ae532846d0ea58b26052bae13dc71f88b0e7 Duration: 1.852034872s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [.........X.XX......XX.XX.XXXXX.X.................X..........] duration: 6.05218086s FAIL (2025-07-29 18:28:45 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M Duration: 6.05220673s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [..........X.XX......XX.XX.XXXXX.X.................X.........] duration: 6.116114446s FAIL (2025-07-29 18:28:20 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M Duration: 6.116140986s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........X.....................................X...........] duration: 1.836682259s FAIL (2025-07-29 18:28:18 UTC) orbiter warning: digest mismatch: after 961b4d588233cdffecfdee9a52135c19a25a5252ae31d78557f60dabdf9ef66b != before 15fa414527dc8e4383d128b95548ae532846d0ea58b26052bae13dc71f88b0e7 Duration: 1.836700986s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.......................XXXXXXXXXXX..........................] duration: 6.151550667s FAIL (2025-07-29 18:28:11 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 6.151578308s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 15 out of 60 is failed (threshold=3) [X..........X.XX......XX.XX.XXXXX.X.................X........] duration: 5.908310061s FAIL (2025-07-29 18:27:52 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M Duration: 5.908338426s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........X.....................................X..........] duration: 1.814580369s FAIL (2025-07-29 18:27:52 UTC) orbiter warning: digest mismatch: after 961b4d588233cdffecfdee9a52135c19a25a5252ae31d78557f60dabdf9ef66b != before 15fa414527dc8e4383d128b95548ae532846d0ea58b26052bae13dc71f88b0e7 Duration: 1.814609945s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [........................XXXXXXXXXXX.........................] duration: 5.689905446s FAIL (2025-07-29 18:27:26 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.689939241s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 15 out of 60 is failed (threshold=3) [.X..........X.XX......XX.XX.XXXXX.X.................X.......] duration: 5.655928687s FAIL (2025-07-29 18:27:26 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M Duration: 5.655953885s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.........................XXXXXXXXXXX........................] duration: 5.833523395s FAIL (2025-07-29 18:27:06 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.833550177s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............X.....................................X.........] duration: 1.941461367s FAIL (2025-07-29 18:27:06 UTC) orbiter warning: digest mismatch: after 961b4d588233cdffecfdee9a52135c19a25a5252ae31d78557f60dabdf9ef66b != before 15fa414527dc8e4383d128b95548ae532846d0ea58b26052bae13dc71f88b0e7 Duration: 1.941526352s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 15 out of 60 is failed (threshold=3) [..X..........X.XX......XX.XX.XXXXX.X.................X......] duration: 5.69156628s FAIL (2025-07-29 18:26:55 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M Duration: 5.691591327s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............X.....................................X........] duration: 1.848464556s FAIL (2025-07-29 18:26:32 UTC) orbiter warning: digest mismatch: after 961b4d588233cdffecfdee9a52135c19a25a5252ae31d78557f60dabdf9ef66b != before 15fa414527dc8e4383d128b95548ae532846d0ea58b26052bae13dc71f88b0e7 Duration: 1.848491098s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..........................XXXXXXXXXXX.......................] duration: 5.805014296s FAIL (2025-07-29 18:26:24 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.805038704s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 15 out of 60 is failed (threshold=3) [...X..........X.XX......XX.XX.XXXXX.X.................X.....] duration: 5.707156545s FAIL (2025-07-29 18:26:13 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M Duration: 5.707179189s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............X.....................................X.......] duration: 1.827065637s FAIL (2025-07-29 18:25:38 UTC) orbiter warning: digest mismatch: after 961b4d588233cdffecfdee9a52135c19a25a5252ae31d78557f60dabdf9ef66b != before 15fa414527dc8e4383d128b95548ae532846d0ea58b26052bae13dc71f88b0e7 Duration: 1.827084264s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...........................XXXXXXXXXXX......................] duration: 5.86507443s FAIL (2025-07-29 18:25:31 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.865098776s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............X.....................................X......] duration: 1.834481699s FAIL (2025-07-29 18:25:08 UTC) orbiter warning: digest mismatch: after 961b4d588233cdffecfdee9a52135c19a25a5252ae31d78557f60dabdf9ef66b != before 15fa414527dc8e4383d128b95548ae532846d0ea58b26052bae13dc71f88b0e7 Duration: 1.834515664s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 15 out of 60 is failed (threshold=3) [....X..........X.XX......XX.XX.XXXXX.X.................X....] duration: 5.778269415s FAIL (2025-07-29 18:25:07 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M Duration: 5.778299253s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [............................XXXXXXXXXXX.....................] duration: 5.680783549s FAIL (2025-07-29 18:24:56 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.680815532s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 15 out of 60 is failed (threshold=3) [.....X..........X.XX......XX.XX.XXXXX.X.................X...] duration: 5.717393053s FAIL (2025-07-29 18:24:44 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M Duration: 5.717418382s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................X.....................................X.....] duration: 1.979289534s FAIL (2025-07-29 18:24:19 UTC) orbiter warning: digest mismatch: after 961b4d588233cdffecfdee9a52135c19a25a5252ae31d78557f60dabdf9ef66b != before 15fa414527dc8e4383d128b95548ae532846d0ea58b26052bae13dc71f88b0e7 Duration: 1.979314421s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.............................XXXXXXXXXXX....................] duration: 5.871229587s FAIL (2025-07-29 18:24:06 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.871259153s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 15 out of 60 is failed (threshold=3) [......X..........X.XX......XX.XX.XXXXX.X.................X..] duration: 5.647655507s FAIL (2025-07-29 18:23:47 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M Duration: 5.647674894s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................X.....................................X....] duration: 1.886808316s FAIL (2025-07-29 18:23:18 UTC) orbiter warning: digest mismatch: after 961b4d588233cdffecfdee9a52135c19a25a5252ae31d78557f60dabdf9ef66b != before 15fa414527dc8e4383d128b95548ae532846d0ea58b26052bae13dc71f88b0e7 Duration: 1.88683678s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..............................XXXXXXXXXXX...................] duration: 5.726269333s FAIL (2025-07-29 18:23:09 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.726296314s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 15 out of 60 is failed (threshold=3) [.......X..........X.XX......XX.XX.XXXXX.X.................X.] duration: 5.712889031s FAIL (2025-07-29 18:22:55 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M Duration: 5.712912505s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...............................XXXXXXXXXXX..................] duration: 5.727937081s FAIL (2025-07-29 18:22:44 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.727961477s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 15 out of 60 is failed (threshold=3) [........X..........X.XX......XX.XX.XXXXX.X.................X] duration: 3.929692487s FAIL (2025-07-29 18:22:30 UTC) orbiter warning: unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M Duration: 3.929714269s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................X.....................................X...] duration: 2.044632254s FAIL (2025-07-29 18:22:03 UTC) orbiter warning: digest mismatch: after 961b4d588233cdffecfdee9a52135c19a25a5252ae31d78557f60dabdf9ef66b != before 15fa414527dc8e4383d128b95548ae532846d0ea58b26052bae13dc71f88b0e7 Duration: 2.044780008s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [................................XXXXXXXXXXX.................] duration: 5.936390263s FAIL (2025-07-29 18:21:46 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.936423287s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [.........X..........X.XX......XX.XX.XXXXX.X.................] duration: 5.909619438s FAIL (2025-07-29 18:21:28 UTC) orbiter warning: uplink: context deadline exceeded Duration: 5.909646471s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.................................XXXXXXXXXXX................] duration: 5.898501361s FAIL (2025-07-29 18:21:25 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.898534364s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................X.....................................X..] duration: 1.99295426s FAIL (2025-07-29 18:21:09 UTC) orbiter warning: digest mismatch: after 961b4d588233cdffecfdee9a52135c19a25a5252ae31d78557f60dabdf9ef66b != before 15fa414527dc8e4383d128b95548ae532846d0ea58b26052bae13dc71f88b0e7 Duration: 1.992989407s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..................................XXXXXXXXXXX...............] duration: 5.766332266s FAIL (2025-07-29 18:21:01 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.766359088s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................X.....................................X.] duration: 1.876936819s FAIL (2025-07-29 18:20:50 UTC) orbiter warning: digest mismatch: after 961b4d588233cdffecfdee9a52135c19a25a5252ae31d78557f60dabdf9ef66b != before 15fa414527dc8e4383d128b95548ae532846d0ea58b26052bae13dc71f88b0e7 Duration: 1.87695893s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [..........X..........X.XX......XX.XX.XXXXX.X................] duration: 5.854533843s FAIL (2025-07-29 18:20:15 UTC) orbiter warning: uplink: context deadline exceeded Duration: 5.854560955s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...................................XXXXXXXXXXX..............] duration: 5.864798803s FAIL (2025-07-29 18:20:08 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.864828679s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................X.....................................X] duration: 1.650310551s FAIL (2025-07-29 18:19:40 UTC) orbiter warning: digest mismatch: after 961b4d588233cdffecfdee9a52135c19a25a5252ae31d78557f60dabdf9ef66b != before 15fa414527dc8e4383d128b95548ae532846d0ea58b26052bae13dc71f88b0e7 Duration: 1.650335739s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 14 out of 60 is failed (threshold=3) [...........X..........X.XX......XX.XX.XXXXX.X...............] duration: 5.996711858s FAIL (2025-07-29 18:19:38 UTC) orbiter warning: uplink: context deadline exceeded Duration: 5.996737027s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [....................................XXXXXXXXXXX.............] duration: 5.735265148s FAIL (2025-07-29 18:18:59 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== 4592: use of closed network connection; write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54518->77.90.31.12:4592: use of closed network connection Jul 29 18:08:45.931909684 (25h22m58.407848149s) [seg_pos={0 0}, upload=24289]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:08:45.931964368 (25h22m58.407902833s) [upload=24289]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:08:45.931968146 (25h22m58.407906612s) [upload=24289]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:08:46.337748236 (25h22m58.813686701s) [upload=24289]: Done uploading segments... Jul 29 18:08:46.337766691 (25h22m58.813705156s) [upload=24289]: Upload finished. err: <nil> Jul 29 18:09:13.411258846 (25h23m25.887197311s) [upload=24302]: Starting upload Jul 29 18:09:13.411327648 (25h23m25.887266113s) [upload=24302]: Done starting upload Jul 29 18:09:13.411448449 (25h23m25.887386914s) [upload=24302]: Uploading segments... Jul 29 18:09:13.411584579 (25h23m25.887523105s) [upload=24302]: Got next segment. Inline: false Jul 29 18:09:13.411619847 (25h23m25.887558312s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 29 18:09:13.411624886 (25h23m25.887563351s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 29 18:09:13.710397165 (25h23m26.186335641s) [seg_pos={0 0}, upload=24302]: Begin upload segment... Jul 29 18:09:13.710437141 (25h23m26.186375607s) [seg_pos={0 0}, upload=24302]: Done begin upload segment. Jul 29 18:09:13.710454736 (25h23m26.186393201s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710487177 (25h23m26.186425642s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710815896 (25h23m26.186754361s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.710946497 (25h23m26.186884962s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711074761 (25h23m26.187013226s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711263392 (25h23m26.187201857s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711364526 (25h23m26.187302991s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711571492 (25h23m26.187509957s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711692223 (25h23m26.187630698s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711842250 (25h23m26.187780715s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.711924327 (25h23m26.187862792s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712130702 (25h23m26.188069167s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712384027 (25h23m26.188322492s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712664794 (25h23m26.188603269s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.712933458 (25h23m26.188871923s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:13.713218333 (25h23m26.189156798s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Uploading piece... Jul 29 18:09:15.846628636 (25h23m28.322567101s) [upload=24302]: Next returned nil segment Jul 29 18:09:15.846643754 (25h23m28.32258221s) [upload=24302]: Waiting for error group managing segments... Jul 29 18:09:16.162057685 (25h23m28.63799615s) [noise=true , address=45.91.116.10:4081 , piece_id=DGV2QCQWDJNB6464AIIHUNODKAJTWRIN6WNVE3YJWGAKM7S6ZQJQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.375105846 (25h23m28.851044301s) [noise=true , address=77.90.31.17:4771 , piece_id=GEQ3ALAZHUYMATD6OE6VU5CVWXZXQGNC3HCNGNW2EDSX5KPPQ6UQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.646830563 (25h23m29.122769028s) [noise=true , address=45.91.116.4:4939 , piece_id=4FIYQZOA2RWW7JI66P3CXEMCLI3JPBOANHGHVQI5MS7T3A2JL75A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.727683722 (25h23m29.203622187s) [noise=true , address=77.90.31.10:4517 , piece_id=6GWSQXOSB65ET7IJMGOZ4E2UDAGROC6GVCA7P64HUYXKRSHPHARQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.832472808 (25h23m29.308411273s) [noise=true , address=77.90.31.17:4772 , piece_id=CAEUN2OQKQUAVI6CKAZWFHU7LYT2RJ66R4QNRPFUU2ZQISB6G52A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:16.899504592 (25h23m29.375443057s) [noise=true , address=45.91.116.12:4225 , piece_id=IYVWIYYWZJUOIU6W6AF4PWAG3J4JSEQMQQRME3KH43Z4A2TH2ROA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.026464017 (25h23m29.502402482s) [noise=true , address=45.91.116.12:4227 , piece_id=SK47FASGK22CBBUHOZLCFVGLHB2FMOQ56WCVKXRNW2TWRXTIBLKA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.067504079 (25h23m29.543442534s) [noise=true , address=45.91.116.7:4009 , piece_id=QSQH2DPGJI2AC3A3JUPNBSCAFW5EHH2TR7O4WXO3LRJEVGH5L6HA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.108526559 (25h23m29.584465024s) [noise=true , address=45.91.116.9:4263 , piece_id=XK6UD776TJBFULO5AXI6K2SM3AY4NCBJXPUQAIOO5OKSGLPJPN4A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.122453906 (25h23m29.598392361s) [noise=true , address=45.91.116.13:4155 , piece_id=6HFG5PEDLN2G4F67VWCTF7C6754OI5P3PM6P5RS6W4SNS6EIZKIQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.251280624 (25h23m29.727219089s) [noise=true , address=77.90.31.19:4843 , piece_id=K3EYVAZ35IEJOG6CYSZ5IMQFAAAHL6VBCKEBIAL5BO3CD7FFPGAA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.276243616 (25h23m29.752182071s) [noise=true , address=77.90.31.11:4553 , piece_id=IRO7MN4IZT6WFQUM2CWUM47Q73ZH23Y6RREZFE6AX7LPGZM5FQMQ, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434347772 (25h23m29.910286227s) [noise=true , address=77.90.31.16:4734 , piece_id=K43GBYIBBWH7MXNDNOB2ELBBWH5AGUKSMXDLXHO6J3XZNERDALYA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: <nil> Jul 29 18:09:17.434373681 (25h23m29.910312146s) [seg_pos={0 0}, upload=24302]: Segment reached optimal threshold of 13 pieces. Jul 29 18:09:17.434478322 (25h23m29.910416777s) [noise=true , address=104.36.82.16:4295 , piece_id=T7SQ6LID2YBECXLOK4JVA6R3ZUJLKCF7UULSOVFQNIMOTIZ6E3TA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434577681 (25h23m29.910516146s) [noise=true , address=77.90.31.12:4589 , piece_id=GLQ5FCTTJYQPOJPGUUI6GXQAGJNHQ3E4RUYWC24MEIAGBPSIKV7A, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv): protocol: expected piece hash; context canceled; EOF Jul 29 18:09:17.434605815 (25h23m29.91054428s) [noise=true , address=45.91.116.10:4083 , piece_id=UTOVFQMF5CWGHIB4GWVEXT535NK72E2DJSW2N6B6YB3GUAILSYSA, seg_pos={0 0}, upload=24302]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37424->45.91.116.10:4083: use of closed network connection Jul 29 18:09:17.434647275 (25h23m29.91058574s) [seg_pos={0 0}, upload=24302]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 29 18:09:17.434722379 (25h23m29.910660844s) [upload=24302]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 29 18:09:17.434726727 (25h23m29.910665192s) [upload=24302]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 29 18:09:17.731175941 (25h23m30.207114406s) [upload=24302]: Done uploading segments... Jul 29 18:09:17.731194988 (25h23m30.207133453s) [upload=24302]: Upload finished. err: <nil> Duration: 5.73528757s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)