Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.719650575s FAIL (2025-08-07 04:31:36 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 46511e3c8ee836342e9b83791f91af91e2d8539c208f329a9198c649a380e28a != before 24e2038caf7e3c9b8dd545c8de8c08ee986034014828d55b4a377a1abf5527b1 Duration: 5.719684511s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X...........................X...............................] duration: 1.858024093s FAIL (2025-08-07 04:31:09 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.858049592s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.789331607s FAIL (2025-08-07 04:30:56 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 46511e3c8ee836342e9b83791f91af91e2d8539c208f329a9198c649a380e28a != before 24e2038caf7e3c9b8dd545c8de8c08ee986034014828d55b4a377a1abf5527b1 Duration: 5.789356895s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.774658754s FAIL (2025-08-07 04:30:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.774679184s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.923920241s FAIL (2025-08-07 04:30:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.923948062s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X...........................X..............................] duration: 1.890633743s FAIL (2025-08-07 04:30:16 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.890660183s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.867031148s FAIL (2025-08-07 04:30:12 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 46511e3c8ee836342e9b83791f91af91e2d8539c208f329a9198c649a380e28a != before 24e2038caf7e3c9b8dd545c8de8c08ee986034014828d55b4a377a1abf5527b1 Duration: 5.867056127s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X...........................X.............................] duration: 2.237279571s FAIL (2025-08-07 04:29:53 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 2.23729984s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.784410431s FAIL (2025-08-07 04:29:46 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 46511e3c8ee836342e9b83791f91af91e2d8539c208f329a9198c649a380e28a != before 24e2038caf7e3c9b8dd545c8de8c08ee986034014828d55b4a377a1abf5527b1 Duration: 5.784438193s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X...........................X............................] duration: 1.828318998s FAIL (2025-08-07 04:29:31 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.828345228s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.787557802s FAIL (2025-08-07 04:29:26 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 46511e3c8ee836342e9b83791f91af91e2d8539c208f329a9198c649a380e28a != before 24e2038caf7e3c9b8dd545c8de8c08ee986034014828d55b4a377a1abf5527b1 Duration: 5.787585696s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.7634445s FAIL (2025-08-07 04:29:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.763476221s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....X...........................X...........................] duration: 1.887004249s FAIL (2025-08-07 04:28:51 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.887032515s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.94562981s FAIL (2025-08-07 04:28:35 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 46511e3c8ee836342e9b83791f91af91e2d8539c208f329a9198c649a380e28a != before 24e2038caf7e3c9b8dd545c8de8c08ee986034014828d55b4a377a1abf5527b1 Duration: 5.945655499s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X....................X......................................] duration: 5.813707508s FAIL (2025-08-07 04:28:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.813733237s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.033672621s FAIL (2025-08-07 04:28:17 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 46511e3c8ee836342e9b83791f91af91e2d8539c208f329a9198c649a380e28a != before 24e2038caf7e3c9b8dd545c8de8c08ee986034014828d55b4a377a1abf5527b1 Duration: 6.033697849s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.585492975s FAIL (2025-08-07 04:27:57 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 46511e3c8ee836342e9b83791f91af91e2d8539c208f329a9198c649a380e28a != before 24e2038caf7e3c9b8dd545c8de8c08ee986034014828d55b4a377a1abf5527b1 Duration: 5.585513204s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X....................X.....................................] duration: 5.773945968s FAIL (2025-08-07 04:27:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.773969374s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....X...........................X..........................] duration: 2.085629132s FAIL (2025-08-07 04:27:48 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 2.085657236s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......X...........................X.........................] duration: 1.845145372s FAIL (2025-08-07 04:27:29 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.845168897s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X....................X....................................] duration: 5.880987749s FAIL (2025-08-07 04:27:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.881014631s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.74618583s FAIL (2025-08-07 04:27:15 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 46511e3c8ee836342e9b83791f91af91e2d8539c208f329a9198c649a380e28a != before 24e2038caf7e3c9b8dd545c8de8c08ee986034014828d55b4a377a1abf5527b1 Duration: 5.746210448s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.798820213s FAIL (2025-08-07 04:26:59 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 46511e3c8ee836342e9b83791f91af91e2d8539c208f329a9198c649a380e28a != before 24e2038caf7e3c9b8dd545c8de8c08ee986034014828d55b4a377a1abf5527b1 Duration: 5.798848636s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X....................X...................................] duration: 5.774567678s FAIL (2025-08-07 04:26:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.774599119s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......X...........................X........................] duration: 1.840759822s FAIL (2025-08-07 04:26:16 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.8407855s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 5.43665489s FAIL (2025-08-07 04:25:45 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 46511e3c8ee836342e9b83791f91af91e2d8539c208f329a9198c649a380e28a != before 24e2038caf7e3c9b8dd545c8de8c08ee986034014828d55b4a377a1abf5527b1 Duration: 5.436681041s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X....................X..................................] duration: 5.929944096s FAIL (2025-08-07 04:25:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.929971238s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........X...........................X.......................] duration: 1.989856675s FAIL (2025-08-07 04:25:13 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.9898912s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X....................X.................................] duration: 6.015478309s FAIL (2025-08-07 04:24:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 6.015509198s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........X...........................X......................] duration: 1.963978321s FAIL (2025-08-07 04:24:01 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.963993822s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X....................X................................] duration: 5.9386164s FAIL (2025-08-07 04:23:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.9386371s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........X...........................X.....................] duration: 1.846707791s FAIL (2025-08-07 04:23:24 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.846738149s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........X...........................X....................] duration: 1.8692436s FAIL (2025-08-07 04:22:59 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.869271433s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X....................X...............................] duration: 5.748829967s FAIL (2025-08-07 04:22:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.748860496s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............X...........................X...................] duration: 1.852771924s FAIL (2025-08-07 04:22:00 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.852799457s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X....................X..............................] duration: 5.766509481s FAIL (2025-08-07 04:21:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.766543225s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X....................X.............................] duration: 5.902971372s FAIL (2025-08-07 04:21:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.902995208s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X....................X............................] duration: 5.753518799s FAIL (2025-08-07 04:21:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.753544438s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............X...........................X..................] duration: 1.815196311s FAIL (2025-08-07 04:21:00 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.815226749s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X....................X...........................] duration: 5.938793878s FAIL (2025-08-07 04:20:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.938821611s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X....................X..........................] duration: 25.25101098s FAIL (2025-08-07 04:20:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 25.251034495s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............X...........................X.................] duration: 2.079463427s FAIL (2025-08-07 04:19:52 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 2.079480018s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X....................X.........................] duration: 22.341767665s FAIL (2025-08-07 04:19:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 22.341795669s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............X...........................X................] duration: 1.845333723s FAIL (2025-08-07 04:19:31 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.845360132s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [................X...........................X...............] duration: 1.809318534s FAIL (2025-08-07 04:18:41 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.809345616s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X....................X........................] duration: 5.821305373s FAIL (2025-08-07 04:18:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.8213304s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................X...........................X..............] duration: 1.905302158s FAIL (2025-08-07 04:18:01 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.905330682s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X....................X.......................] duration: 5.819591731s FAIL (2025-08-07 04:17:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== WatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136236350 (13h49m24.074652199s) [noise=true , address=77.90.31.10:4519 , piece_id=XI4WMZRV2Y4HJ2FEJN6G6ZQSIXKPUI2ZSZQAZRW4M4XRTMNGIHPQ, seg_pos={0 0}, upload=13245]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:16.136320912 (13h49m24.074736771s) [seg_pos={0 0}, upload=13245]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:16.136399292 (13h49m24.074815141s) [upload=13245]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:16.136403420 (13h49m24.074819269s) [upload=13245]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:16.427080155 (13h49m24.365496004s) [upload=13245]: Done uploading segments... Aug 7 03:58:16.427097879 (13h49m24.365513729s) [upload=13245]: Upload finished. err: <nil> Aug 7 03:58:30.184956648 (13h49m38.123372497s) [upload=13250]: Starting upload Aug 7 03:58:30.185004009 (13h49m38.123419848s) [upload=13250]: Done starting upload Aug 7 03:58:30.185205905 (13h49m38.123621744s) [upload=13250]: Uploading segments... Aug 7 03:58:30.185236314 (13h49m38.123652163s) [upload=13250]: Got next segment. Inline: false Aug 7 03:58:30.185257183 (13h49m38.123673022s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 03:58:30.185260930 (13h49m38.123676779s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 03:58:30.454001938 (13h49m38.392417787s) [seg_pos={0 0}, upload=13250]: Begin upload segment... Aug 7 03:58:30.454044699 (13h49m38.392460548s) [seg_pos={0 0}, upload=13250]: Done begin upload segment. Aug 7 03:58:30.454066982 (13h49m38.392482831s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454100586 (13h49m38.392516435s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454427672 (13h49m38.392843511s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454588760 (13h49m38.393004609s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454692479 (13h49m38.393108328s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454908060 (13h49m38.393323909s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.454963637 (13h49m38.393379486s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455232281 (13h49m38.39364813s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455255596 (13h49m38.393671445s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455479334 (13h49m38.393895183s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455506566 (13h49m38.393922405s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455787894 (13h49m38.394203743s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.455835545 (13h49m38.394251384s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456019998 (13h49m38.394435848s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.456223638 (13h49m38.394639487s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:30.457172833 (13h49m38.395588682s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Uploading piece... Aug 7 03:58:32.206529945 (13h49m40.144945785s) [upload=13250]: Next returned nil segment Aug 7 03:58:32.206545354 (13h49m40.144961203s) [upload=13250]: Waiting for error group managing segments... Aug 7 03:58:32.482974069 (13h49m40.421389908s) [noise=true , address=45.91.116.13:4154 , piece_id=VUW4IGZYBTU747MSA53EEMTQSWXJLJABZWTF6KM2DEUQ6C4ZU3MA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.546641532 (13h49m40.485057371s) [noise=true , address=45.91.116.13:4155 , piece_id=77RZ5H3YTCF5YNRDGTIZEQNZ56RAOD2M7TNT7LAEOO3WIYZZLBIA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.547859872 (13h49m40.486275711s) [noise=true , address=45.91.116.14:4191 , piece_id=XN2CJFUBQOFX6KBUQGXLTGWO5AMPSJJCDYF6ITBGKFDWBMHO2HTQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.555284067 (13h49m40.493699906s) [noise=true , address=77.90.31.18:4806 , piece_id=CYXMDPQR5VYF3FKEINVYRQK6OJPMIBTE5WRYU7GN5DPDD7YIRCRA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.561331598 (13h49m40.499747447s) [noise=true , address=45.91.116.6:4973 , piece_id=QB52CSFAPSCQS2HIRVP5QQQJDQAAOMEIEFM537ZYL7BLIOC4LXQA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.562139544 (13h49m40.500555393s) [noise=true , address=45.91.116.11:4119 , piece_id=YRWRT5QTBJL23QFZ2KLJD6TFGZTFE3746CIQUGFS6RNGOXFDEH2A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.566721550 (13h49m40.505137399s) [noise=true , address=45.91.116.7:4010 , piece_id=WLWBH3RUDCH27SNZS3LU3DCF3TI7M66JXHLUBBBXIFFH5NIF63HA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.567470363 (13h49m40.505886212s) [noise=true , address=45.91.116.10:4083 , piece_id=SHSBQD7DFD2G67W3VA6EQSN7ZKPRF7VFDUK22UJ2KFV6TWK56GBQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.576776648 (13h49m40.515192497s) [noise=true , address=77.90.31.10:4517 , piece_id=P7G4OODCQPYGYUM5PA5LPCASJ45NHORQ35WXZ5PWIIANSI4A52OQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.580990250 (13h49m40.519406099s) [noise=true , address=77.90.31.17:4772 , piece_id=FG27XTGPGLPPUVHTJYZVPF3XHEH4G6QGLKXAQY4BV7MJSE7CSOSA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.584837120 (13h49m40.523252959s) [noise=true , address=104.36.82.7:4909 , piece_id=4CZDSCQSBXRRQBWCEOUWJT2C6TLVWY2I6IVERDWAVWZOTEZN6Z7A, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.585356173 (13h49m40.523772042s) [noise=true , address=104.36.82.16:4293 , piece_id=SREUVOSA7477XN3VHUF24OFQLEERODGS63Q7MCFFVUQSLLHZCMRQ, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601057214 (13h49m40.539473053s) [noise=true , address=104.36.82.11:4115 , piece_id=4AQY54Z4E7JIOORSCUPMPODOWWDFR5IYC2WFV3B3K3T3XD3RQVBA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: <nil> Aug 7 03:58:32.601086330 (13h49m40.539502219s) [seg_pos={0 0}, upload=13250]: Segment reached optimal threshold of 13 pieces. Aug 7 03:58:32.601212661 (13h49m40.53962851s) [noise=true , address=45.91.116.8:4045 , piece_id=WGQLWXHFCF4ONW7ZDUGERFZJHUWVW2MZC77DDVCATTD6RYEHRDOA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601285240 (13h49m40.539701089s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7TOOYKHVZ5RRAVU5NDNOTO7NIIBUAQAO4ZOC7JZL5LVAYLZQPA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601415780 (13h49m40.539831629s) [noise=true , address=45.91.116.6:4974 , piece_id=BNJAVQ7IJHPQMRP4L2RKMFS44ROE3ERZXXCGKVMZREW2X5MDH3RA, seg_pos={0 0}, upload=13250]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ): protocol: expected piece hash; context canceled; EOF Aug 7 03:58:32.601463891 (13h49m40.53987973s) [seg_pos={0 0}, upload=13250]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 03:58:32.601549495 (13h49m40.539965334s) [upload=13250]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 03:58:32.601553784 (13h49m40.539969633s) [upload=13250]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 03:58:32.906871209 (13h49m40.845287058s) [upload=13250]: Done uploading segments... Aug 7 03:58:32.906889855 (13h49m40.845305704s) [upload=13250]: Upload finished. err: <nil> Duration: 5.819619202s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................X...........................X.............] duration: 1.942758972s FAIL (2025-08-07 04:17:03 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.942781406s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................X...........................X............] duration: 1.802938409s FAIL (2025-08-07 04:16:25 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 8bded342a915d4f0c07a8cfde5842bc8aa16148028158d2cc091561675a691f7 != before 8e2888fdf9937ccd28d428972cd1fe99109edfbfcc431d44ae938629e820e791 Duration: 1.802961152s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)