Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................X...........................X.............] duration: 1.942758972s FAIL (2025-08-07 04:17:03 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.942781406s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................X...........................X............] duration: 1.802938409s FAIL (2025-08-07 04:16:25 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.802961152s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X....................X......................] duration: 5.949611846s FAIL (2025-08-07 04:16:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.9496453s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X....................X.....................] duration: 5.902607281s FAIL (2025-08-07 04:15:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.90263827s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................X...........................X...........] duration: 1.831721269s FAIL (2025-08-07 04:15:19 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.831764771s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................X...........................X..........] duration: 1.880353616s FAIL (2025-08-07 04:14:50 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.880379094s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................X....................X....................] duration: 6.02269247s FAIL (2025-08-07 04:14:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 6.022722036s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................X....................X...................] duration: 6.065312063s FAIL (2025-08-07 04:13:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 6.065332681s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......................X...........................X.........] duration: 1.858422104s FAIL (2025-08-07 04:13:49 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.858453443s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................X....................X..................] duration: 17.611618962s FAIL (2025-08-07 04:13:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 17.61164447s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................X...........................X........] duration: 1.860100443s FAIL (2025-08-07 04:13:09 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.860160046s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................X....................X.................] duration: 19.127890374s FAIL (2025-08-07 04:12:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 19.127916213s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................X...........................X.......] duration: 1.859945798s FAIL (2025-08-07 04:12:01 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.859984632s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................X...........................X......] duration: 1.992422962s FAIL (2025-08-07 04:11:22 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.992443551s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X.....................X....................X................] duration: 6.41668206s FAIL (2025-08-07 04:10:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 6.416710874s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................X...........................X.....] duration: 1.876383899s FAIL (2025-08-07 04:10:29 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.876409618s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X.....................X....................X...............] duration: 6.155542498s FAIL (2025-08-07 04:10:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 6.155571624s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........................X...........................X....] duration: 1.957315475s FAIL (2025-08-07 04:09:55 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.957373625s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................X...........................X...] duration: 1.924654893s FAIL (2025-08-07 04:09:39 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.924695842s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X.....................X....................X..............] duration: 6.146893101s FAIL (2025-08-07 04:09:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 6.146926072s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............................X...........................X..] duration: 1.955424535s FAIL (2025-08-07 04:08:35 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.955449653s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X.....................X....................X.............] duration: 6.133181613s FAIL (2025-08-07 04:08:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 6.133201521s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............................X...........................X.] duration: 1.860000522s FAIL (2025-08-07 04:07:27 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.860048072s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............................X...........................X] duration: 1.607689454s FAIL (2025-08-07 04:07:04 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.607716585s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X.....................X....................X............] duration: 6.539092996s FAIL (2025-08-07 04:06:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 6.539123074s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.734100164s FAIL (2025-08-07 04:06:27 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 5.734137946s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.5679346s FAIL (2025-08-07 04:06:11 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 5.567959367s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X.....................X....................X...........] duration: 6.041457299s FAIL (2025-08-07 04:06:04 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 6.041491875s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 1.834285932s FAIL (2025-08-07 04:05:59 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/1K") Duration: 1.834307403s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.800404171s FAIL (2025-08-07 04:05:50 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 5.800425982s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X.....................X....................X..........] duration: 5.963542499s FAIL (2025-08-07 04:05:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.963571855s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 1.836309867s FAIL (2025-08-07 04:05:41 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/1K") Duration: 1.836335716s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.673125965s FAIL (2025-08-07 04:05:28 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 5.673152717s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X.....................X....................X.........] duration: 5.880025398s FAIL (2025-08-07 04:05:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.880056907s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.643277527s FAIL (2025-08-07 04:05:05 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 5.643308427s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 1.923315854s FAIL (2025-08-07 04:05:01 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/1K") Duration: 1.923335571s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X.....................X....................X........] duration: 28.004817251s FAIL (2025-08-07 04:04:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 28.004836938s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.048969333s FAIL (2025-08-07 04:04:20 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 6.048997887s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.953869558s FAIL (2025-08-07 04:03:58 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 5.953896048s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 1.841855558s FAIL (2025-08-07 04:03:51 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/1K") Duration: 1.841881157s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X.....................X....................X.......] duration: 5.808077842s FAIL (2025-08-07 04:03:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.808102818s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 1.902892084s FAIL (2025-08-07 04:03:32 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/1K") Duration: 1.902919198s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.015972741s FAIL (2025-08-07 04:03:03 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/1K") Duration: 2.0159988s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X.....................X....................X......] duration: 5.970684042s FAIL (2025-08-07 04:02:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.970701406s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.573010347s FAIL (2025-08-07 04:02:48 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 5.573037279s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X.....................X....................X.....] duration: 12.547847363s FAIL (2025-08-07 04:02:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 12.547870728s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 1.972808164s FAIL (2025-08-07 04:02:17 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/1K") Duration: 1.972839424s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 1.939155689s FAIL (2025-08-07 04:01:57 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/1K") Duration: 1.939181498s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X.....................X....................X....] duration: 5.748576119s FAIL (2025-08-07 04:01:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.748615506s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.728200416s FAIL (2025-08-07 04:01:35 UTC) orbiter warning: operation: DownloadObject uplink: object not found ("orbiter_us_0/4M") Duration: 5.728219253s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)