Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 414904 reports (Page 283 of 8299)
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
57 out of 60 is failed (threshold=6) [XXXXXX.XX.XXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
5.001329412s
trace:
https://jaeger.dev.storj.io/trace/31d68308b9cb9a50
FAIL (2026-01-06 16:47:23 UTC)
orbiter warning: context deadline exceeded
Duration: 5.001351526s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
21 out of 60 is failed (threshold=3) [XX..X.....XXX........X.......X..XX.....XX.X...X.XXX.XX..X.X.]
duration:
5.868917236s
trace:
https://jaeger.dev.storj.io/trace/56c54e6ec12c955a
FAIL (2026-01-06 16:47:10 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: 5DT2XKJG5VVT2SFDNZXCYGFDC2XCR3HPU3IO7VETL2SJRODB7NYQ) context deadline exceeded; piecestore: (Node ID: 155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, Piece ID: NGCRBP3DJDFVWC4QQXZVM2FCCG3VGZDQUQQ3XCTTVPJKAEJUCTXA) context deadline exceeded
======LOGS======
Jan  6 16:46:13.929589347 (769h17m13.800672382s) [upload=673061]: Starting upload
Jan  6 16:46:13.929705077 (769h17m13.800788122s) [upload=673061]: Done starting upload
Jan  6 16:46:13.929792444 (769h17m13.800875489s) [upload=673061]: Uploading segments...
Jan  6 16:46:13.930027310 (769h17m13.801110355s) [upload=673061]: Got next segment. Inline: false
Jan  6 16:46:13.930094328 (769h17m13.801177373s) [upload=673061]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan  6 16:46:13.930100921 (769h17m13.801183966s) [upload=673061]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan  6 16:46:14.178245576 (769h17m14.049328611s) [seg_pos={0 0}, upload=673061]: Begin upload segment...
Jan  6 16:46:14.178302264 (769h17m14.04938531s) [seg_pos={0 0}, upload=673061]: Done begin upload segment.
Jan  6 16:46:14.178342070 (769h17m14.049425105s) [noise=true , address=145.79.227.6:4257    , piece_id=FLMRPGBSEDV6GSNDE25RUGQVXZGEFQUJDH4EVARKN7SIM2YWDVTQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.178453352 (769h17m14.049536397s) [noise=true , address=145.79.227.51:4409   , piece_id=M3GYAKANPLDGVAVYMQR7T62QEL7E4UVYJZSXKUT33ISFKU2O4PEQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.179197258 (769h17m14.050280303s) [noise=true , address=77.90.31.14:4663     , piece_id=NGCRBP3DJDFVWC4QQXZVM2FCCG3VGZDQUQQ3XCTTVPJKAEJUCTXA, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.179687612 (769h17m14.050770657s) [noise=true , address=77.90.31.19:4841     , piece_id=UDTELXSNRCPEXCNEYPIUC24FU3IQ4MKUN4XD7TGQA5JML4JGM6QA, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.180201991 (769h17m14.051285036s) [noise=true , address=45.91.116.13:4153    , piece_id=F34JXABDIK7Z2QCQ6SS7URTAMPND6XC3IL4WCTAKQXMQQA34WXHA, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.180635828 (769h17m14.051718873s) [noise=true , address=145.79.227.39:4013   , piece_id=JME3GBIDBYXRDIVY4GI5HEUUDKSOHTTRZVFNTJ2HAKQ5G2KRZ4NQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.181068031 (769h17m14.052151076s) [noise=true , address=145.79.227.3:4149    , piece_id=4GKBQUOAQWPH5HIOU2GHBJKUNNGLIACQNRXAZMOU77PJNGN4WHFQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.181515793 (769h17m14.052598838s) [noise=true , address=145.79.227.41:4085   , piece_id=AQJBMYNYI6YPHZHPG7RS5EZNCM4NAAWWKS42V7ZKVB7CA62TS7CA, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.182139361 (769h17m14.053222406s) [noise=true , address=145.79.227.15:4873   , piece_id=VIYEEEHJKHMZQGYE5JEWEI5SZ2ZCRE7RDUHP3O5EDAQU66YBZ75Q, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.183828026 (769h17m14.054911071s) [noise=true , address=145.79.227.14:4839   , piece_id=AZAIFL25PYWBAOAYZGJDRFGGEEGRIYAGK4I7AXDGNHZODA64EIOA, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.186019839 (769h17m14.057102884s) [noise=true , address=145.79.227.50:4375   , piece_id=FQNYVRBLHCBT2YBKZKZUFRE2SYS7UXGJOLLGZ3BTUOJ52VE2HERQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.187787175 (769h17m14.05887022s) [noise=true , address=45.91.116.10:4081    , piece_id=BPUKOD4LOQVPYPIRGUT7DHWELPL5DKMSKUAJGDXDIZABOEL4WPFQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.189771523 (769h17m14.060854559s) [noise=true , address=145.79.227.7:4293    , piece_id=GCYNFSAQ3A3O3K3ZZJNK7EA62UKKAAYBRYHFQMLZB3W4VFMNCGNQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.192210187 (769h17m14.063293222s) [noise=true , address=145.79.227.33:4869   , piece_id=HA45Z73MU37KWMROM5C7ZF7C77HN4ZUGSA7MLGJEEADOXUZAGPUA, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.201052264 (769h17m14.072135299s) [noise=true , address=145.79.227.30:4761   , piece_id=6CCLDIUQFUT6RGNYRAXVEXL6E7ROYRRHPJ6OT2D77AYGJQ5H76GQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.201689287 (769h17m14.072772332s) [noise=true , address=45.91.116.14:4189    , piece_id=5DT2XKJG5VVT2SFDNZXCYGFDC2XCR3HPU3IO7VETL2SJRODB7NYQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:15.884686982 (769h17m15.755770017s) [upload=673061]: Next returned nil segment
Jan  6 16:46:15.884699916 (769h17m15.755782961s) [upload=673061]: Waiting for error group managing segments...
Jan  6 16:46:16.163971723 (769h17m16.035054768s) [noise=true , address=145.79.227.51:4409   , piece_id=M3GYAKANPLDGVAVYMQR7T62QEL7E4UVYJZSXKUT33ISFKU2O4PEQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.168074815 (769h17m16.03915785s) [noise=true , address=145.79.227.41:4085   , piece_id=AQJBMYNYI6YPHZHPG7RS5EZNCM4NAAWWKS42V7ZKVB7CA62TS7CA, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.168778675 (769h17m16.03986173s) [noise=true , address=145.79.227.15:4873   , piece_id=VIYEEEHJKHMZQGYE5JEWEI5SZ2ZCRE7RDUHP3O5EDAQU66YBZ75Q, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.187813812 (769h17m16.058896847s) [noise=true , address=145.79.227.50:4375   , piece_id=FQNYVRBLHCBT2YBKZKZUFRE2SYS7UXGJOLLGZ3BTUOJ52VE2HERQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.188444462 (769h17m16.059527508s) [noise=true , address=145.79.227.39:4013   , piece_id=JME3GBIDBYXRDIVY4GI5HEUUDKSOHTTRZVFNTJ2HAKQ5G2KRZ4NQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.203119726 (769h17m16.074202761s) [noise=true , address=145.79.227.6:4257    , piece_id=FLMRPGBSEDV6GSNDE25RUGQVXZGEFQUJDH4EVARKN7SIM2YWDVTQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.203651389 (769h17m16.074734424s) [noise=true , address=145.79.227.7:4293    , piece_id=GCYNFSAQ3A3O3K3ZZJNK7EA62UKKAAYBRYHFQMLZB3W4VFMNCGNQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.207415326 (769h17m16.078498361s) [noise=true , address=145.79.227.14:4839   , piece_id=AZAIFL25PYWBAOAYZGJDRFGGEEGRIYAGK4I7AXDGNHZODA64EIOA, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.310290606 (769h17m16.181373641s) [noise=true , address=145.79.227.3:4149    , piece_id=4GKBQUOAQWPH5HIOU2GHBJKUNNGLIACQNRXAZMOU77PJNGN4WHFQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.554127028 (769h17m16.425210063s) [noise=true , address=77.90.31.19:4841     , piece_id=UDTELXSNRCPEXCNEYPIUC24FU3IQ4MKUN4XD7TGQA5JML4JGM6QA, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.666202837 (769h17m16.537285883s) [noise=true , address=45.91.116.14:4189    , piece_id=5DT2XKJG5VVT2SFDNZXCYGFDC2XCR3HPU3IO7VETL2SJRODB7NYQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.675630950 (769h17m16.546713995s) [noise=true , address=45.91.116.13:4153    , piece_id=F34JXABDIK7Z2QCQ6SS7URTAMPND6XC3IL4WCTAKQXMQQA34WXHA, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:17.260831291 (769h17m17.131914336s) [noise=true , address=77.90.31.14:4663     , piece_id=NGCRBP3DJDFVWC4QQXZVM2FCCG3VGZDQUQQ3XCTTVPJKAEJUCTXA, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:17.260855167 (769h17m17.131938212s) [seg_pos={0 0}, upload=673061]: Segment reached optimal threshold of 13 pieces.
Jan  6 16:46:17.261157622 (769h17m17.132240687s) [noise=true , address=145.79.227.30:4761   , piece_id=6CCLDIUQFUT6RGNYRAXVEXL6E7ROYRRHPJ6OT2D77AYGJQ5H76GQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE): protocol: expected piece hash; context canceled; EOF
Jan  6 16:46:17.261167712 (769h17m17.132250757s) [noise=true , address=145.79.227.30:4761   , piece_id=6CCLDIUQFUT6RGNYRAXVEXL6E7ROYRRHPJ6OT2D77AYGJQ5H76GQ, seg_pos={0 0}, upload=673061]: Long tail cancellation due to optimal threshold
Jan  6 16:46:17.261255839 (769h17m17.132338874s) [noise=true , address=145.79.227.33:4869   , piece_id=HA45Z73MU37KWMROM5C7ZF7C77HN4ZUGSA7MLGJEEADOXUZAGPUA, seg_pos={0 0}, upload=673061]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CQxjq65r6xVTmNRurSQPzKxj4BoR6cJ8Uq29sTrdh6uXm5sq): protocol: expected piece hash; context canceled; EOF
Jan  6 16:46:17.261270417 (769h17m17.132353453s) [noise=true , address=145.79.227.33:4869   , piece_id=HA45Z73MU37KWMROM5C7ZF7C77HN4ZUGSA7MLGJEEADOXUZAGPUA, seg_pos={0 0}, upload=673061]: Long tail cancellation due to optimal threshold
Jan  6 16:46:17.261418439 (769h17m17.132501474s) [noise=true , address=45.91.116.10:4081    , piece_id=BPUKOD4LOQVPYPIRGUT7DHWELPL5DKMSKUAJGDXDIZABOEL4WPFQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF
Jan  6 16:46:17.261424901 (769h17m17.132507936s) [noise=true , address=45.91.116.10:4081    , piece_id=BPUKOD4LOQVPYPIRGUT7DHWELPL5DKMSKUAJGDXDIZABOEL4WPFQ, seg_pos={0 0}, upload=673061]: Long tail cancellation due to optimal threshold
Jan  6 16:46:17.261440511 (769h17m17.132523547s) [seg_pos={0 0}, upload=673061]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan  6 16:46:17.261561051 (769h17m17.132644086s) [upload=673061]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:46:17.261566451 (769h17m17.132654645s) [upload=673061]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:46:17.563105607 (769h17m17.434188662s) [upload=673061]: Done uploading segments...
Jan  6 16:46:17.563130986 (769h17m17.434214041s) [upload=673061]: Upload finished. err: <nil>
Duration: 5.868948476s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
12 out of 60 is failed (threshold=3) [..................X.X....X.X.X....X.X...X....XXX...........X]
duration:
30.002092469s
trace:
https://jaeger.dev.storj.io/trace/58c0b7fa2e1d819
FAIL (2026-01-06 16:47:10 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.002115664s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
22 out of 60 is failed (threshold=3) [XXX..X.....XXX........X.......X..XX.....XX.X...X.XXX.XX..X.X]
duration:
30.002861833s
trace:
https://jaeger.dev.storj.io/trace/56c54e6ec12c955a
FAIL (2026-01-06 16:46:43 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, Piece ID: 5DT2XKJG5VVT2SFDNZXCYGFDC2XCR3HPU3IO7VETL2SJRODB7NYQ) context deadline exceeded; piecestore: (Node ID: 155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, Piece ID: NGCRBP3DJDFVWC4QQXZVM2FCCG3VGZDQUQQ3XCTTVPJKAEJUCTXA) context deadline exceeded
======LOGS======
Jan  6 16:46:13.929589347 (769h17m13.800672382s) [upload=673061]: Starting upload
Jan  6 16:46:13.929705077 (769h17m13.800788122s) [upload=673061]: Done starting upload
Jan  6 16:46:13.929792444 (769h17m13.800875489s) [upload=673061]: Uploading segments...
Jan  6 16:46:13.930027310 (769h17m13.801110355s) [upload=673061]: Got next segment. Inline: false
Jan  6 16:46:13.930094328 (769h17m13.801177373s) [upload=673061]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan  6 16:46:13.930100921 (769h17m13.801183966s) [upload=673061]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan  6 16:46:14.178245576 (769h17m14.049328611s) [seg_pos={0 0}, upload=673061]: Begin upload segment...
Jan  6 16:46:14.178302264 (769h17m14.04938531s) [seg_pos={0 0}, upload=673061]: Done begin upload segment.
Jan  6 16:46:14.178342070 (769h17m14.049425105s) [noise=true , address=145.79.227.6:4257    , piece_id=FLMRPGBSEDV6GSNDE25RUGQVXZGEFQUJDH4EVARKN7SIM2YWDVTQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.178453352 (769h17m14.049536397s) [noise=true , address=145.79.227.51:4409   , piece_id=M3GYAKANPLDGVAVYMQR7T62QEL7E4UVYJZSXKUT33ISFKU2O4PEQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.179197258 (769h17m14.050280303s) [noise=true , address=77.90.31.14:4663     , piece_id=NGCRBP3DJDFVWC4QQXZVM2FCCG3VGZDQUQQ3XCTTVPJKAEJUCTXA, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.179687612 (769h17m14.050770657s) [noise=true , address=77.90.31.19:4841     , piece_id=UDTELXSNRCPEXCNEYPIUC24FU3IQ4MKUN4XD7TGQA5JML4JGM6QA, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.180201991 (769h17m14.051285036s) [noise=true , address=45.91.116.13:4153    , piece_id=F34JXABDIK7Z2QCQ6SS7URTAMPND6XC3IL4WCTAKQXMQQA34WXHA, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.180635828 (769h17m14.051718873s) [noise=true , address=145.79.227.39:4013   , piece_id=JME3GBIDBYXRDIVY4GI5HEUUDKSOHTTRZVFNTJ2HAKQ5G2KRZ4NQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.181068031 (769h17m14.052151076s) [noise=true , address=145.79.227.3:4149    , piece_id=4GKBQUOAQWPH5HIOU2GHBJKUNNGLIACQNRXAZMOU77PJNGN4WHFQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.181515793 (769h17m14.052598838s) [noise=true , address=145.79.227.41:4085   , piece_id=AQJBMYNYI6YPHZHPG7RS5EZNCM4NAAWWKS42V7ZKVB7CA62TS7CA, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.182139361 (769h17m14.053222406s) [noise=true , address=145.79.227.15:4873   , piece_id=VIYEEEHJKHMZQGYE5JEWEI5SZ2ZCRE7RDUHP3O5EDAQU66YBZ75Q, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.183828026 (769h17m14.054911071s) [noise=true , address=145.79.227.14:4839   , piece_id=AZAIFL25PYWBAOAYZGJDRFGGEEGRIYAGK4I7AXDGNHZODA64EIOA, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.186019839 (769h17m14.057102884s) [noise=true , address=145.79.227.50:4375   , piece_id=FQNYVRBLHCBT2YBKZKZUFRE2SYS7UXGJOLLGZ3BTUOJ52VE2HERQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.187787175 (769h17m14.05887022s) [noise=true , address=45.91.116.10:4081    , piece_id=BPUKOD4LOQVPYPIRGUT7DHWELPL5DKMSKUAJGDXDIZABOEL4WPFQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.189771523 (769h17m14.060854559s) [noise=true , address=145.79.227.7:4293    , piece_id=GCYNFSAQ3A3O3K3ZZJNK7EA62UKKAAYBRYHFQMLZB3W4VFMNCGNQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.192210187 (769h17m14.063293222s) [noise=true , address=145.79.227.33:4869   , piece_id=HA45Z73MU37KWMROM5C7ZF7C77HN4ZUGSA7MLGJEEADOXUZAGPUA, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.201052264 (769h17m14.072135299s) [noise=true , address=145.79.227.30:4761   , piece_id=6CCLDIUQFUT6RGNYRAXVEXL6E7ROYRRHPJ6OT2D77AYGJQ5H76GQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:14.201689287 (769h17m14.072772332s) [noise=true , address=45.91.116.14:4189    , piece_id=5DT2XKJG5VVT2SFDNZXCYGFDC2XCR3HPU3IO7VETL2SJRODB7NYQ, seg_pos={0 0}, upload=673061]: Uploading piece...
Jan  6 16:46:15.884686982 (769h17m15.755770017s) [upload=673061]: Next returned nil segment
Jan  6 16:46:15.884699916 (769h17m15.755782961s) [upload=673061]: Waiting for error group managing segments...
Jan  6 16:46:16.163971723 (769h17m16.035054768s) [noise=true , address=145.79.227.51:4409   , piece_id=M3GYAKANPLDGVAVYMQR7T62QEL7E4UVYJZSXKUT33ISFKU2O4PEQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.168074815 (769h17m16.03915785s) [noise=true , address=145.79.227.41:4085   , piece_id=AQJBMYNYI6YPHZHPG7RS5EZNCM4NAAWWKS42V7ZKVB7CA62TS7CA, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.168778675 (769h17m16.03986173s) [noise=true , address=145.79.227.15:4873   , piece_id=VIYEEEHJKHMZQGYE5JEWEI5SZ2ZCRE7RDUHP3O5EDAQU66YBZ75Q, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.187813812 (769h17m16.058896847s) [noise=true , address=145.79.227.50:4375   , piece_id=FQNYVRBLHCBT2YBKZKZUFRE2SYS7UXGJOLLGZ3BTUOJ52VE2HERQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.188444462 (769h17m16.059527508s) [noise=true , address=145.79.227.39:4013   , piece_id=JME3GBIDBYXRDIVY4GI5HEUUDKSOHTTRZVFNTJ2HAKQ5G2KRZ4NQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.203119726 (769h17m16.074202761s) [noise=true , address=145.79.227.6:4257    , piece_id=FLMRPGBSEDV6GSNDE25RUGQVXZGEFQUJDH4EVARKN7SIM2YWDVTQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.203651389 (769h17m16.074734424s) [noise=true , address=145.79.227.7:4293    , piece_id=GCYNFSAQ3A3O3K3ZZJNK7EA62UKKAAYBRYHFQMLZB3W4VFMNCGNQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.207415326 (769h17m16.078498361s) [noise=true , address=145.79.227.14:4839   , piece_id=AZAIFL25PYWBAOAYZGJDRFGGEEGRIYAGK4I7AXDGNHZODA64EIOA, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.310290606 (769h17m16.181373641s) [noise=true , address=145.79.227.3:4149    , piece_id=4GKBQUOAQWPH5HIOU2GHBJKUNNGLIACQNRXAZMOU77PJNGN4WHFQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.554127028 (769h17m16.425210063s) [noise=true , address=77.90.31.19:4841     , piece_id=UDTELXSNRCPEXCNEYPIUC24FU3IQ4MKUN4XD7TGQA5JML4JGM6QA, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.666202837 (769h17m16.537285883s) [noise=true , address=45.91.116.14:4189    , piece_id=5DT2XKJG5VVT2SFDNZXCYGFDC2XCR3HPU3IO7VETL2SJRODB7NYQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:16.675630950 (769h17m16.546713995s) [noise=true , address=45.91.116.13:4153    , piece_id=F34JXABDIK7Z2QCQ6SS7URTAMPND6XC3IL4WCTAKQXMQQA34WXHA, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:17.260831291 (769h17m17.131914336s) [noise=true , address=77.90.31.14:4663     , piece_id=NGCRBP3DJDFVWC4QQXZVM2FCCG3VGZDQUQQ3XCTTVPJKAEJUCTXA, seg_pos={0 0}, upload=673061]: Done uploading piece. err: <nil>
Jan  6 16:46:17.260855167 (769h17m17.131938212s) [seg_pos={0 0}, upload=673061]: Segment reached optimal threshold of 13 pieces.
Jan  6 16:46:17.261157622 (769h17m17.132240687s) [noise=true , address=145.79.227.30:4761   , piece_id=6CCLDIUQFUT6RGNYRAXVEXL6E7ROYRRHPJ6OT2D77AYGJQ5H76GQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12aik37v1FXxLLPx5EoA5EWszHVsRUz7kAriBYUbWEtN15555xE): protocol: expected piece hash; context canceled; EOF
Jan  6 16:46:17.261167712 (769h17m17.132250757s) [noise=true , address=145.79.227.30:4761   , piece_id=6CCLDIUQFUT6RGNYRAXVEXL6E7ROYRRHPJ6OT2D77AYGJQ5H76GQ, seg_pos={0 0}, upload=673061]: Long tail cancellation due to optimal threshold
Jan  6 16:46:17.261255839 (769h17m17.132338874s) [noise=true , address=145.79.227.33:4869   , piece_id=HA45Z73MU37KWMROM5C7ZF7C77HN4ZUGSA7MLGJEEADOXUZAGPUA, seg_pos={0 0}, upload=673061]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CQxjq65r6xVTmNRurSQPzKxj4BoR6cJ8Uq29sTrdh6uXm5sq): protocol: expected piece hash; context canceled; EOF
Jan  6 16:46:17.261270417 (769h17m17.132353453s) [noise=true , address=145.79.227.33:4869   , piece_id=HA45Z73MU37KWMROM5C7ZF7C77HN4ZUGSA7MLGJEEADOXUZAGPUA, seg_pos={0 0}, upload=673061]: Long tail cancellation due to optimal threshold
Jan  6 16:46:17.261418439 (769h17m17.132501474s) [noise=true , address=45.91.116.10:4081    , piece_id=BPUKOD4LOQVPYPIRGUT7DHWELPL5DKMSKUAJGDXDIZABOEL4WPFQ, seg_pos={0 0}, upload=673061]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG): protocol: expected piece hash; context canceled; EOF
Jan  6 16:46:17.261424901 (769h17m17.132507936s) [noise=true , address=45.91.116.10:4081    , piece_id=BPUKOD4LOQVPYPIRGUT7DHWELPL5DKMSKUAJGDXDIZABOEL4WPFQ, seg_pos={0 0}, upload=673061]: Long tail cancellation due to optimal threshold
Jan  6 16:46:17.261440511 (769h17m17.132523547s) [seg_pos={0 0}, upload=673061]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan  6 16:46:17.261561051 (769h17m17.132644086s) [upload=673061]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:46:17.261566451 (769h17m17.132654645s) [upload=673061]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:46:17.563105607 (769h17m17.434188662s) [upload=673061]: Done uploading segments...
Jan  6 16:46:17.563130986 (769h17m17.434214041s) [upload=673061]: Upload finished. err: <nil>
Duration: 30.002884255s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
57 out of 60 is failed (threshold=6) [XXXXXXX.XX.XXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
5.001082435s
trace:
https://jaeger.dev.storj.io/trace/2f19e3507a2b68b0
FAIL (2026-01-06 16:46:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001109867s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
2 out of 60 is failed (threshold=3) [...................X......X.................................]
duration:
27.470656016s
trace:
https://jaeger.dev.storj.io/trace/d5e2d8b8127d900
FAIL (2026-01-06 16:46:30 UTC)
orbiter warning: context deadline exceeded
Duration: 27.470686171s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
13 out of 60 is failed (threshold=3) [..X..X.X.X........X.....XX......X..........XX...X.X......X..]
duration:
15.360404886s
trace:
https://jaeger.dev.storj.io/trace/5dfad5b7812907ca
FAIL (2026-01-06 16:46:17 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
ing piece...
Jan  6 16:44:46.321352518 (769h15m46.192435553s) [noise=true , address=145.79.227.45:4250   , piece_id=HPEJNGEZKKZTQGFDMREMI2HT2YFOWIH2XYKCI525UHGOIHZPD7VA, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.321612172 (769h15m46.192695207s) [noise=true , address=145.79.227.13:4808   , piece_id=UEMCSOO4WUMC2VQYHTKX7DH6XZMAD44HOTTUYMID34AZXJEDGXDA, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.322147592 (769h15m46.193230627s) [noise=true , address=145.79.227.23:4573   , piece_id=DAKGS66HRC7OAAGDMI7XU4GBYT4MYRWBOFCH7RF2A4KP3VUIQA2Q, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.322463263 (769h15m46.193546298s) [noise=true , address=145.79.227.7:4302    , piece_id=EDW7TST2H5EDOW2R3KTYI7IZGWUFEVJ3YA7KBBZZ42ESO56LGBGA, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.322752634 (769h15m46.193835669s) [noise=true , address=145.79.227.33:4894   , piece_id=D7SPQ5SNK5SPFQWZE54LAJWG3TRKV4NUOBGANXMF2YB37ASIAANA, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.323156573 (769h15m46.194239608s) [noise=true , address=145.79.227.4:4198    , piece_id=E6RB3OBCCHSFBRIHHHR56E7M5RZ23TNYYQPPQ5UKE4RP2MW42J3Q, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.323514775 (769h15m46.19459781s) [noise=true , address=145.79.227.40:4053   , piece_id=LUTH3OGSOTO6OYSCTHVIDAT57QDG32FYGXBTMQCAR5F3PXJU5OMQ, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.323917182 (769h15m46.195000217s) [noise=true , address=145.79.227.29:4971   , piece_id=ROIKGGPDHXZ2XCSVIAP4ETUD6ACOKIIVG7EVYCYO6AL47G7G6ZIQ, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.324251077 (769h15m46.195334112s) [noise=true , address=64.49.13.138:4486    , piece_id=TDFOABDUDWDEMPDFFUIR6J7H6OCYFETTBJHCCIPGJUO5NMIJEGUA, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.324473541 (769h15m46.195556576s) [noise=true , address=77.90.31.12:4601     , piece_id=BLD52KPVFQQLCK2MVZ2EHJHLRMIVK47DNUE7JRTGPXBD4R4WSFWQ, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.324568541 (769h15m46.195651576s) [noise=true , address=45.91.116.12:4235    , piece_id=C3GJTOVEHMK6CQ3TWC7NGO4CXMGCQ4VC4WO6EMN646TVFZLFTSCQ, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.324878532 (769h15m46.195961567s) [noise=true , address=145.79.227.36:4529   , piece_id=YNZEOHMFBHOMAIZBV7FPRTBXSA3THOQ2R5XEND4RRGL4PNLM2W5A, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.324899021 (769h15m46.195982056s) [noise=true , address=145.79.227.20:4455   , piece_id=ZHAXGWI47TGWTZA4XU6XSZTD3GRCSG6AUU7N5S7VQXTAJAVDIDJQ, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.325203371 (769h15m46.196286406s) [noise=true , address=145.79.227.49:4362   , piece_id=OD3PZGZU2NBXSFGEJYVWADPJHGSC33ERBTKGRSVDNKGPC66YX77A, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:47.757963144 (769h15m47.629046179s) [upload=673036]: Next returned nil segment
Jan  6 16:44:47.757975688 (769h15m47.629058723s) [upload=673036]: Waiting for error group managing segments...
Jan  6 16:44:48.175355824 (769h15m48.046438859s) [noise=true , address=145.79.227.36:4529   , piece_id=YNZEOHMFBHOMAIZBV7FPRTBXSA3THOQ2R5XEND4RRGL4PNLM2W5A, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.189871995 (769h15m48.06095503s) [noise=true , address=145.79.227.3:4160    , piece_id=MVK7OHYWBQL3U6ICJFFUCLGDMIA436VRAVYHDNZRX6TZ7KME25RA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.207607777 (769h15m48.078690822s) [noise=true , address=64.49.13.138:4486    , piece_id=TDFOABDUDWDEMPDFFUIR6J7H6OCYFETTBJHCCIPGJUO5NMIJEGUA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.227981511 (769h15m48.099064556s) [noise=true , address=145.79.227.23:4573   , piece_id=DAKGS66HRC7OAAGDMI7XU4GBYT4MYRWBOFCH7RF2A4KP3VUIQA2Q, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.247897795 (769h15m48.11898084s) [noise=true , address=145.79.227.20:4455   , piece_id=ZHAXGWI47TGWTZA4XU6XSZTD3GRCSG6AUU7N5S7VQXTAJAVDIDJQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.349641861 (769h15m48.220724906s) [noise=true , address=145.79.227.13:4808   , piece_id=UEMCSOO4WUMC2VQYHTKX7DH6XZMAD44HOTTUYMID34AZXJEDGXDA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.782145845 (769h15m48.65322888s) [noise=true , address=145.79.227.4:4198    , piece_id=E6RB3OBCCHSFBRIHHHR56E7M5RZ23TNYYQPPQ5UKE4RP2MW42J3Q, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.997160406 (769h15m48.868243451s) [noise=true , address=45.91.116.12:4235    , piece_id=C3GJTOVEHMK6CQ3TWC7NGO4CXMGCQ4VC4WO6EMN646TVFZLFTSCQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.027918673 (769h15m48.899001708s) [noise=true , address=193.149.164.143:4695 , piece_id=Y5VL4E75Q5ULJCK4CLWBR43WOFXRO226UTHHH4JABUIVOT2TFJOA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.084408278 (769h15m48.955491313s) [noise=true , address=77.90.31.12:4601     , piece_id=BLD52KPVFQQLCK2MVZ2EHJHLRMIVK47DNUE7JRTGPXBD4R4WSFWQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.128152033 (769h15m48.999235079s) [noise=true , address=145.79.227.29:4971   , piece_id=ROIKGGPDHXZ2XCSVIAP4ETUD6ACOKIIVG7EVYCYO6AL47G7G6ZIQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.133368196 (769h15m49.004451231s) [noise=true , address=145.79.227.44:4216   , piece_id=2PBU2YR4KAMTVSVEG4NRLL6IZPQMX3DJCBXVG3K72LZQMLDQWQSA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.134414158 (769h15m49.005497203s) [noise=true , address=145.79.227.40:4053   , piece_id=LUTH3OGSOTO6OYSCTHVIDAT57QDG32FYGXBTMQCAR5F3PXJU5OMQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.212147304 (769h15m49.083230349s) [noise=true , address=145.79.227.49:4362   , piece_id=OD3PZGZU2NBXSFGEJYVWADPJHGSC33ERBTKGRSVDNKGPC66YX77A, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.212173132 (769h15m49.083256178s) [seg_pos={0 0}, upload=673036]: Segment reached optimal threshold of 14 pieces.
Jan  6 16:44:49.212410895 (769h15m49.08349393s) [noise=true , address=145.79.227.45:4250   , piece_id=HPEJNGEZKKZTQGFDMREMI2HT2YFOWIH2XYKCI525UHGOIHZPD7VA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD): protocol: write tcp 188.166.225.170:58078->145.79.227.45:4250: use of closed network connection; write tcp 188.166.225.170:58078->145.79.227.45:4250: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58078->145.79.227.45:4250: use of closed network connection
Jan  6 16:44:49.212419001 (769h15m49.083502036s) [noise=true , address=145.79.227.45:4250   , piece_id=HPEJNGEZKKZTQGFDMREMI2HT2YFOWIH2XYKCI525UHGOIHZPD7VA, seg_pos={0 0}, upload=673036]: Long tail cancellation due to optimal threshold
Jan  6 16:44:49.212661392 (769h15m49.083744427s) [noise=true , address=45.91.116.8:4053     , piece_id=WKZWJ4EBW2DR5JIROXLBNFUNBEAMGDYZSKHIAAT4YD6F4YSDW3SQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LA2misujs8mqKZRnYqV4kCZoG46KZaxjyYGqT1HdpzEFhyBf6): protocol: read tcp 188.166.225.170:37938->45.91.116.8:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan  6 16:44:49.212690538 (769h15m49.083773583s) [noise=true , address=45.91.116.8:4053     , piece_id=WKZWJ4EBW2DR5JIROXLBNFUNBEAMGDYZSKHIAAT4YD6F4YSDW3SQ, seg_pos={0 0}, upload=673036]: Long tail cancellation due to optimal threshold
Jan  6 16:44:49.212707120 (769h15m49.083790155s) [noise=true , address=145.79.227.7:4302    , piece_id=EDW7TST2H5EDOW2R3KTYI7IZGWUFEVJ3YA7KBBZZ42ESO56LGBGA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288AZgzLsbgYuXzMt5exDkjAyUUokGHFRD8dK9f88uDep7oVxH): protocol: write tcp 188.166.225.170:52152->145.79.227.7:4302: use of closed network connection; write tcp 188.166.225.170:52152->145.79.227.7:4302: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52152->145.79.227.7:4302: use of closed network connection
Jan  6 16:44:49.212715395 (769h15m49.083798441s) [noise=true , address=145.79.227.7:4302    , piece_id=EDW7TST2H5EDOW2R3KTYI7IZGWUFEVJ3YA7KBBZZ42ESO56LGBGA, seg_pos={0 0}, upload=673036]: Long tail cancellation due to optimal threshold
Jan  6 16:44:49.212904215 (769h15m49.08398726s) [noise=true , address=145.79.227.33:4894   , piece_id=D7SPQ5SNK5SPFQWZE54LAJWG3TRKV4NUOBGANXMF2YB37ASIAANA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF
Jan  6 16:44:49.212923271 (769h15m49.084006306s) [noise=true , address=145.79.227.33:4894   , piece_id=D7SPQ5SNK5SPFQWZE54LAJWG3TRKV4NUOBGANXMF2YB37ASIAANA, seg_pos={0 0}, upload=673036]: Long tail cancellation due to optimal threshold
Jan  6 16:44:49.212943820 (769h15m49.084026865s) [seg_pos={0 0}, upload=673036]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jan  6 16:44:49.213046986 (769h15m49.084130031s) [upload=673036]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:44:49.213051696 (769h15m49.084134731s) [upload=673036]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:44:49.517610893 (769h15m49.388693949s) [upload=673036]: Done uploading segments...
Jan  6 16:44:49.517643054 (769h15m49.388726089s) [upload=673036]: Upload finished. err: <nil>
Duration: 15.360419064s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)
antiblip:
2 out of 60 is failed (threshold=3) [.............................................XX.............]
duration:
21.205415271s
trace:
https://jaeger.dev.storj.io/trace/4b3b841c440bfc62
FAIL (2026-01-06 16:46:16 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
JXEZNOOX75ENDV6EBXU4NR4HW4GTZMVA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913312874 (769h5m40.784395909s) [noise=true , address=81.171.14.70:4035    , piece_id=BRXIJD7JX7L37MWMTVPL5KSESQUQEZNRXLBLL26XDMYIJSCRUD5A, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913698277 (769h5m40.784781322s) [noise=true , address=81.171.14.70:4030    , piece_id=WV5BULPUQSAE7DUO2FWY74KF3MIHSQXYC673GNJ7XUBQXXM6KHEQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913862180 (769h5m40.784945215s) [noise=true , address=89.149.201.147:4004  , piece_id=NMPCYNSE3YVIAVDCWRO4VP6IN4DKZ2P4YIFYI6ZDTEU75QSFC4NQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.914346662 (769h5m40.785429707s) [noise=true , address=185.167.204.205:4035 , piece_id=HPHYG44QNKJYFSXEBH6FPYLFXLO6OUIVGDYL5GXJMP3PDNFUZEQA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.914842456 (769h5m40.785925491s) [noise=true , address=81.171.14.70:4034    , piece_id=5LGDXPCHMOQPZGJAEH3ZSPF64RSTVAJRQFZBXVWP5D3Y36YFLUUQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.915307461 (769h5m40.786390506s) [noise=true , address=89.149.201.147:4019  , piece_id=LS3NXHELDI7H6VCTHO6LAGRDDG46ST4IF2XOFZDZ6EK756K5ACHQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.915793817 (769h5m40.786876863s) [noise=true , address=185.167.204.206:4067 , piece_id=GLITDKTEKYV4CFGL3TJXWJDFOBYPFX2UTNJFPUCZDJM3HOWVFS4Q, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.916279973 (769h5m40.787363018s) [noise=true , address=89.149.201.147:4014  , piece_id=KNV6OXYOC2H7LVZCW4ZWQRYRN5ZCMII3OO3FAS3YYQ22574KUCRQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.916759336 (769h5m40.787842371s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.920089677 (769h5m40.791172722s) [noise=true , address=185.167.204.205:4014 , piece_id=XUCDQV4VW4ZZFTSZV2YEDAO5GQFP2ASW6BJFFC735UWEVIR3RNXA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:41.741185400 (769h5m41.612268445s) [upload=672894]: Next returned nil segment
Jan  6 16:34:41.741201020 (769h5m41.612284055s) [upload=672894]: Waiting for error group managing segments...
Jan  6 16:34:42.136024704 (769h5m42.00710774s) [noise=true , address=185.167.204.205:4036 , piece_id=4XKAIZPEJZAVAS7P2QILKY7XIDIIRIRWH7HTN5OVXUHAL2Y3H4SQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.137413950 (769h5m42.008496995s) [noise=true , address=185.167.204.206:4051 , piece_id=XWKJM6ZIHISLSESJ2ZQQQMZTGMK5MHH6L4ZD5RXJ3SMAVASVVVOA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.137527506 (769h5m42.008610541s) [noise=true , address=185.167.204.206:4049 , piece_id=4T665GS7XTBVEHUVT6SFIQW4F3TTBPDKZ57PKQHHHWWKC4P4JQPQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.138183485 (769h5m42.00926653s) [noise=true , address=185.167.204.206:4048 , piece_id=ZGQYUYRK5PK24UF4PYT67FHNPR4GVFDCJ2OEGRFENLB2GYR2LFVQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.139902609 (769h5m42.010985644s) [noise=true , address=185.167.204.206:4056 , piece_id=KYHDW5RLJTZCURXCFIP7HN3ZYLPPHK7FTO6IIGQ5VJJUPF4ICQXA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.140277843 (769h5m42.011360878s) [noise=true , address=185.167.204.206:4067 , piece_id=GLITDKTEKYV4CFGL3TJXWJDFOBYPFX2UTNJFPUCZDJM3HOWVFS4Q, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.142394303 (769h5m42.013477348s) [noise=true , address=185.167.204.205:4021 , piece_id=U4X3NZRT7MHG2JTNRVDRYIC72UDMHUQUCY4VFW7JFVLLVACZN7OA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.179529514 (769h5m42.050612549s) [noise=true , address=185.167.204.206:4053 , piece_id=HBFCBJCOLPJLNE4INHP4JXEZNOOX75ENDV6EBXU4NR4HW4GTZMVA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.194387988 (769h5m42.065471023s) [noise=true , address=185.167.204.205:4035 , piece_id=HPHYG44QNKJYFSXEBH6FPYLFXLO6OUIVGDYL5GXJMP3PDNFUZEQA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.197860059 (769h5m42.068943094s) [noise=true , address=185.167.204.206:4064 , piece_id=JGCXE3QUORVKLRIWEXFZUCT4XUF3DRVA35ZKQU4DQODLZBOH5I4A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.827615690 (769h5m42.698698725s) [noise=true , address=81.171.14.70:4041    , piece_id=NLGMNQJDFISSAHBX3EC6WMWJLP74FMR4MRPUIX4GCCCQRG5RRRCQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.917676689 (769h5m42.788759725s) [noise=true , address=81.171.14.70:4034    , piece_id=5LGDXPCHMOQPZGJAEH3ZSPF64RSTVAJRQFZBXVWP5D3Y36YFLUUQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.932213350 (769h5m42.803296385s) [noise=true , address=81.171.14.70:4030    , piece_id=WV5BULPUQSAE7DUO2FWY74KF3MIHSQXYC673GNJ7XUBQXXM6KHEQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.956051121 (769h5m42.827134196s) [noise=true , address=81.171.14.70:4031    , piece_id=OJKJHNVWTPDITZC5CTQNB7UFPIVZDPE54ZXS4LWRM3XHYAB6443A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.005259852 (769h5m42.876342887s) [noise=true , address=185.167.204.205:4019 , piece_id=I7OP7AQS2DC3C7GZWG5RVJSLDKAZD2434WMBPRYMZFVAJYJG3JHQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.052708792 (769h5m42.923791837s) [noise=true , address=81.171.14.70:4046    , piece_id=OV3PMCD6UIFNSQNTCOS3GEQC4676KFJOUH5ZX576OFOYC5EXCZPQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.639749858 (769h5m43.510832903s) [noise=true , address=89.149.201.147:4004  , piece_id=NMPCYNSE3YVIAVDCWRO4VP6IN4DKZ2P4YIFYI6ZDTEU75QSFC4NQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.719270598 (769h5m43.590353643s) [noise=true , address=89.149.201.147:4019  , piece_id=LS3NXHELDI7H6VCTHO6LAGRDDG46ST4IF2XOFZDZ6EK756K5ACHQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.892052420 (769h5m43.763135466s) [noise=true , address=81.171.14.70:4035    , piece_id=BRXIJD7JX7L37MWMTVPL5KSESQUQEZNRXLBLL26XDMYIJSCRUD5A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.901787498 (769h5m43.772870533s) [noise=true , address=81.171.14.70:4039    , piece_id=VFXOSS5DRL43RDJGPLSH3667HWKAF5ZNC2CF5BA7TDR4NSCG75HA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.019887208 (769h5m43.890970243s) [noise=true , address=185.167.204.206:4050 , piece_id=KSCVNGXUGFV7XRNU5MFIKLLQGZ5BNHTIXQLVHWPMITZOQUA23DGQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.510337921 (769h5m44.381420957s) [noise=true , address=185.167.204.205:4028 , piece_id=QPB4EPPEENW277S4SBIOCWWMBLIE6GKCGEM3TV3MXN6SYFMQAI7Q, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.692198242 (769h5m44.563281277s) [noise=true , address=89.149.201.147:4010  , piece_id=C57ORVBIZE4QUWCVG3OVG3KW2Z2GYVWUXEOYG5R53KV2WUQLR7JQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.394574949 (769h5m46.265657984s) [noise=true , address=89.149.201.147:4014  , piece_id=KNV6OXYOC2H7LVZCW4ZWQRYRN5ZCMII3OO3FAS3YYQ22574KUCRQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.543864889 (769h5m46.414947934s) [noise=true , address=185.167.204.205:4014 , piece_id=XUCDQV4VW4ZZFTSZV2YEDAO5GQFP2ASW6BJFFC735UWEVIR3RNXA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.846238946 (769h5m46.717321982s) [noise=true , address=185.167.204.205:4006 , piece_id=W3XXEI7MG37VJHHK2CECNKIIHB7WSQABGVB5W5ZGAWSTFQ4NQMEQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.846258273 (769h5m46.717341308s) [seg_pos={0 0}, upload=672894]: Segment reached optimal threshold of 26 pieces.
Jan  6 16:34:46.846448986 (769h5m46.717532021s) [noise=true , address=185.167.204.206:4072 , piece_id=EJ4XCLNDZETIFIKE3FK2Q4NA35KT5LOTNGE253MKUCAIGH4FZD5A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v): protocol: expected piece hash; context canceled; EOF
Jan  6 16:34:46.846454998 (769h5m46.717538033s) [noise=true , address=185.167.204.206:4072 , piece_id=EJ4XCLNDZETIFIKE3FK2Q4NA35KT5LOTNGE253MKUCAIGH4FZD5A, seg_pos={0 0}, upload=672894]: Long tail cancellation due to optimal threshold
Jan  6 16:34:46.846579175 (769h5m46.71766221s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK): protocol: expected piece hash; context canceled; EOF
Jan  6 16:34:46.846595475 (769h5m46.71767851s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Long tail cancellation due to optimal threshold
Jan  6 16:34:46.846620453 (769h5m46.717703488s) [seg_pos={0 0}, upload=672894]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan  6 16:34:46.846729380 (769h5m46.717812415s) [upload=672894]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:34:46.846734600 (769h5m46.717817635s) [upload=672894]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:34:47.079572108 (769h5m46.950655143s) [upload=672894]: Done uploading segments...
Jan  6 16:34:47.079590854 (769h5m46.950673889s) [upload=672894]: Upload finished. err: <nil>
Duration: 21.205440671s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
13 out of 60 is failed (threshold=3) [...X..X.X.X........X.....XX......X..........XX...X.X......X.]
duration:
15.889355012s
trace:
https://jaeger.dev.storj.io/trace/5dfad5b7812907ca
FAIL (2026-01-06 16:46:01 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
ing piece...
Jan  6 16:44:46.321352518 (769h15m46.192435553s) [noise=true , address=145.79.227.45:4250   , piece_id=HPEJNGEZKKZTQGFDMREMI2HT2YFOWIH2XYKCI525UHGOIHZPD7VA, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.321612172 (769h15m46.192695207s) [noise=true , address=145.79.227.13:4808   , piece_id=UEMCSOO4WUMC2VQYHTKX7DH6XZMAD44HOTTUYMID34AZXJEDGXDA, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.322147592 (769h15m46.193230627s) [noise=true , address=145.79.227.23:4573   , piece_id=DAKGS66HRC7OAAGDMI7XU4GBYT4MYRWBOFCH7RF2A4KP3VUIQA2Q, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.322463263 (769h15m46.193546298s) [noise=true , address=145.79.227.7:4302    , piece_id=EDW7TST2H5EDOW2R3KTYI7IZGWUFEVJ3YA7KBBZZ42ESO56LGBGA, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.322752634 (769h15m46.193835669s) [noise=true , address=145.79.227.33:4894   , piece_id=D7SPQ5SNK5SPFQWZE54LAJWG3TRKV4NUOBGANXMF2YB37ASIAANA, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.323156573 (769h15m46.194239608s) [noise=true , address=145.79.227.4:4198    , piece_id=E6RB3OBCCHSFBRIHHHR56E7M5RZ23TNYYQPPQ5UKE4RP2MW42J3Q, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.323514775 (769h15m46.19459781s) [noise=true , address=145.79.227.40:4053   , piece_id=LUTH3OGSOTO6OYSCTHVIDAT57QDG32FYGXBTMQCAR5F3PXJU5OMQ, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.323917182 (769h15m46.195000217s) [noise=true , address=145.79.227.29:4971   , piece_id=ROIKGGPDHXZ2XCSVIAP4ETUD6ACOKIIVG7EVYCYO6AL47G7G6ZIQ, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.324251077 (769h15m46.195334112s) [noise=true , address=64.49.13.138:4486    , piece_id=TDFOABDUDWDEMPDFFUIR6J7H6OCYFETTBJHCCIPGJUO5NMIJEGUA, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.324473541 (769h15m46.195556576s) [noise=true , address=77.90.31.12:4601     , piece_id=BLD52KPVFQQLCK2MVZ2EHJHLRMIVK47DNUE7JRTGPXBD4R4WSFWQ, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.324568541 (769h15m46.195651576s) [noise=true , address=45.91.116.12:4235    , piece_id=C3GJTOVEHMK6CQ3TWC7NGO4CXMGCQ4VC4WO6EMN646TVFZLFTSCQ, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.324878532 (769h15m46.195961567s) [noise=true , address=145.79.227.36:4529   , piece_id=YNZEOHMFBHOMAIZBV7FPRTBXSA3THOQ2R5XEND4RRGL4PNLM2W5A, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.324899021 (769h15m46.195982056s) [noise=true , address=145.79.227.20:4455   , piece_id=ZHAXGWI47TGWTZA4XU6XSZTD3GRCSG6AUU7N5S7VQXTAJAVDIDJQ, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.325203371 (769h15m46.196286406s) [noise=true , address=145.79.227.49:4362   , piece_id=OD3PZGZU2NBXSFGEJYVWADPJHGSC33ERBTKGRSVDNKGPC66YX77A, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:47.757963144 (769h15m47.629046179s) [upload=673036]: Next returned nil segment
Jan  6 16:44:47.757975688 (769h15m47.629058723s) [upload=673036]: Waiting for error group managing segments...
Jan  6 16:44:48.175355824 (769h15m48.046438859s) [noise=true , address=145.79.227.36:4529   , piece_id=YNZEOHMFBHOMAIZBV7FPRTBXSA3THOQ2R5XEND4RRGL4PNLM2W5A, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.189871995 (769h15m48.06095503s) [noise=true , address=145.79.227.3:4160    , piece_id=MVK7OHYWBQL3U6ICJFFUCLGDMIA436VRAVYHDNZRX6TZ7KME25RA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.207607777 (769h15m48.078690822s) [noise=true , address=64.49.13.138:4486    , piece_id=TDFOABDUDWDEMPDFFUIR6J7H6OCYFETTBJHCCIPGJUO5NMIJEGUA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.227981511 (769h15m48.099064556s) [noise=true , address=145.79.227.23:4573   , piece_id=DAKGS66HRC7OAAGDMI7XU4GBYT4MYRWBOFCH7RF2A4KP3VUIQA2Q, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.247897795 (769h15m48.11898084s) [noise=true , address=145.79.227.20:4455   , piece_id=ZHAXGWI47TGWTZA4XU6XSZTD3GRCSG6AUU7N5S7VQXTAJAVDIDJQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.349641861 (769h15m48.220724906s) [noise=true , address=145.79.227.13:4808   , piece_id=UEMCSOO4WUMC2VQYHTKX7DH6XZMAD44HOTTUYMID34AZXJEDGXDA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.782145845 (769h15m48.65322888s) [noise=true , address=145.79.227.4:4198    , piece_id=E6RB3OBCCHSFBRIHHHR56E7M5RZ23TNYYQPPQ5UKE4RP2MW42J3Q, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.997160406 (769h15m48.868243451s) [noise=true , address=45.91.116.12:4235    , piece_id=C3GJTOVEHMK6CQ3TWC7NGO4CXMGCQ4VC4WO6EMN646TVFZLFTSCQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.027918673 (769h15m48.899001708s) [noise=true , address=193.149.164.143:4695 , piece_id=Y5VL4E75Q5ULJCK4CLWBR43WOFXRO226UTHHH4JABUIVOT2TFJOA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.084408278 (769h15m48.955491313s) [noise=true , address=77.90.31.12:4601     , piece_id=BLD52KPVFQQLCK2MVZ2EHJHLRMIVK47DNUE7JRTGPXBD4R4WSFWQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.128152033 (769h15m48.999235079s) [noise=true , address=145.79.227.29:4971   , piece_id=ROIKGGPDHXZ2XCSVIAP4ETUD6ACOKIIVG7EVYCYO6AL47G7G6ZIQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.133368196 (769h15m49.004451231s) [noise=true , address=145.79.227.44:4216   , piece_id=2PBU2YR4KAMTVSVEG4NRLL6IZPQMX3DJCBXVG3K72LZQMLDQWQSA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.134414158 (769h15m49.005497203s) [noise=true , address=145.79.227.40:4053   , piece_id=LUTH3OGSOTO6OYSCTHVIDAT57QDG32FYGXBTMQCAR5F3PXJU5OMQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.212147304 (769h15m49.083230349s) [noise=true , address=145.79.227.49:4362   , piece_id=OD3PZGZU2NBXSFGEJYVWADPJHGSC33ERBTKGRSVDNKGPC66YX77A, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.212173132 (769h15m49.083256178s) [seg_pos={0 0}, upload=673036]: Segment reached optimal threshold of 14 pieces.
Jan  6 16:44:49.212410895 (769h15m49.08349393s) [noise=true , address=145.79.227.45:4250   , piece_id=HPEJNGEZKKZTQGFDMREMI2HT2YFOWIH2XYKCI525UHGOIHZPD7VA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD): protocol: write tcp 188.166.225.170:58078->145.79.227.45:4250: use of closed network connection; write tcp 188.166.225.170:58078->145.79.227.45:4250: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58078->145.79.227.45:4250: use of closed network connection
Jan  6 16:44:49.212419001 (769h15m49.083502036s) [noise=true , address=145.79.227.45:4250   , piece_id=HPEJNGEZKKZTQGFDMREMI2HT2YFOWIH2XYKCI525UHGOIHZPD7VA, seg_pos={0 0}, upload=673036]: Long tail cancellation due to optimal threshold
Jan  6 16:44:49.212661392 (769h15m49.083744427s) [noise=true , address=45.91.116.8:4053     , piece_id=WKZWJ4EBW2DR5JIROXLBNFUNBEAMGDYZSKHIAAT4YD6F4YSDW3SQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LA2misujs8mqKZRnYqV4kCZoG46KZaxjyYGqT1HdpzEFhyBf6): protocol: read tcp 188.166.225.170:37938->45.91.116.8:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan  6 16:44:49.212690538 (769h15m49.083773583s) [noise=true , address=45.91.116.8:4053     , piece_id=WKZWJ4EBW2DR5JIROXLBNFUNBEAMGDYZSKHIAAT4YD6F4YSDW3SQ, seg_pos={0 0}, upload=673036]: Long tail cancellation due to optimal threshold
Jan  6 16:44:49.212707120 (769h15m49.083790155s) [noise=true , address=145.79.227.7:4302    , piece_id=EDW7TST2H5EDOW2R3KTYI7IZGWUFEVJ3YA7KBBZZ42ESO56LGBGA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288AZgzLsbgYuXzMt5exDkjAyUUokGHFRD8dK9f88uDep7oVxH): protocol: write tcp 188.166.225.170:52152->145.79.227.7:4302: use of closed network connection; write tcp 188.166.225.170:52152->145.79.227.7:4302: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52152->145.79.227.7:4302: use of closed network connection
Jan  6 16:44:49.212715395 (769h15m49.083798441s) [noise=true , address=145.79.227.7:4302    , piece_id=EDW7TST2H5EDOW2R3KTYI7IZGWUFEVJ3YA7KBBZZ42ESO56LGBGA, seg_pos={0 0}, upload=673036]: Long tail cancellation due to optimal threshold
Jan  6 16:44:49.212904215 (769h15m49.08398726s) [noise=true , address=145.79.227.33:4894   , piece_id=D7SPQ5SNK5SPFQWZE54LAJWG3TRKV4NUOBGANXMF2YB37ASIAANA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF
Jan  6 16:44:49.212923271 (769h15m49.084006306s) [noise=true , address=145.79.227.33:4894   , piece_id=D7SPQ5SNK5SPFQWZE54LAJWG3TRKV4NUOBGANXMF2YB37ASIAANA, seg_pos={0 0}, upload=673036]: Long tail cancellation due to optimal threshold
Jan  6 16:44:49.212943820 (769h15m49.084026865s) [seg_pos={0 0}, upload=673036]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jan  6 16:44:49.213046986 (769h15m49.084130031s) [upload=673036]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:44:49.213051696 (769h15m49.084134731s) [upload=673036]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:44:49.517610893 (769h15m49.388693949s) [upload=673036]: Done uploading segments...
Jan  6 16:44:49.517643054 (769h15m49.388726089s) [upload=673036]: Upload finished. err: <nil>
Duration: 15.88936908s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
11 out of 60 is failed (threshold=3) [...................X.X....X.X.X....X.X...X....XXX...........]
duration:
25.359853804s
trace:
https://jaeger.dev.storj.io/trace/73dd0c7413e4b545
FAIL (2026-01-06 16:45:57 UTC)
orbiter warning: context deadline exceeded
Duration: 25.359889993s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
13 out of 60 is failed (threshold=3) [....X..X.X.X........X.....XX......X..........XX...X.X......X]
duration:
1m0.005899751s
trace:
https://jaeger.dev.storj.io/trace/5dfad5b7812907ca
FAIL (2026-01-06 16:45:45 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: context deadline exceeded
======LOGS======
ing piece...
Jan  6 16:44:46.321352518 (769h15m46.192435553s) [noise=true , address=145.79.227.45:4250   , piece_id=HPEJNGEZKKZTQGFDMREMI2HT2YFOWIH2XYKCI525UHGOIHZPD7VA, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.321612172 (769h15m46.192695207s) [noise=true , address=145.79.227.13:4808   , piece_id=UEMCSOO4WUMC2VQYHTKX7DH6XZMAD44HOTTUYMID34AZXJEDGXDA, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.322147592 (769h15m46.193230627s) [noise=true , address=145.79.227.23:4573   , piece_id=DAKGS66HRC7OAAGDMI7XU4GBYT4MYRWBOFCH7RF2A4KP3VUIQA2Q, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.322463263 (769h15m46.193546298s) [noise=true , address=145.79.227.7:4302    , piece_id=EDW7TST2H5EDOW2R3KTYI7IZGWUFEVJ3YA7KBBZZ42ESO56LGBGA, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.322752634 (769h15m46.193835669s) [noise=true , address=145.79.227.33:4894   , piece_id=D7SPQ5SNK5SPFQWZE54LAJWG3TRKV4NUOBGANXMF2YB37ASIAANA, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.323156573 (769h15m46.194239608s) [noise=true , address=145.79.227.4:4198    , piece_id=E6RB3OBCCHSFBRIHHHR56E7M5RZ23TNYYQPPQ5UKE4RP2MW42J3Q, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.323514775 (769h15m46.19459781s) [noise=true , address=145.79.227.40:4053   , piece_id=LUTH3OGSOTO6OYSCTHVIDAT57QDG32FYGXBTMQCAR5F3PXJU5OMQ, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.323917182 (769h15m46.195000217s) [noise=true , address=145.79.227.29:4971   , piece_id=ROIKGGPDHXZ2XCSVIAP4ETUD6ACOKIIVG7EVYCYO6AL47G7G6ZIQ, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.324251077 (769h15m46.195334112s) [noise=true , address=64.49.13.138:4486    , piece_id=TDFOABDUDWDEMPDFFUIR6J7H6OCYFETTBJHCCIPGJUO5NMIJEGUA, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.324473541 (769h15m46.195556576s) [noise=true , address=77.90.31.12:4601     , piece_id=BLD52KPVFQQLCK2MVZ2EHJHLRMIVK47DNUE7JRTGPXBD4R4WSFWQ, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.324568541 (769h15m46.195651576s) [noise=true , address=45.91.116.12:4235    , piece_id=C3GJTOVEHMK6CQ3TWC7NGO4CXMGCQ4VC4WO6EMN646TVFZLFTSCQ, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.324878532 (769h15m46.195961567s) [noise=true , address=145.79.227.36:4529   , piece_id=YNZEOHMFBHOMAIZBV7FPRTBXSA3THOQ2R5XEND4RRGL4PNLM2W5A, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.324899021 (769h15m46.195982056s) [noise=true , address=145.79.227.20:4455   , piece_id=ZHAXGWI47TGWTZA4XU6XSZTD3GRCSG6AUU7N5S7VQXTAJAVDIDJQ, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:46.325203371 (769h15m46.196286406s) [noise=true , address=145.79.227.49:4362   , piece_id=OD3PZGZU2NBXSFGEJYVWADPJHGSC33ERBTKGRSVDNKGPC66YX77A, seg_pos={0 0}, upload=673036]: Uploading piece...
Jan  6 16:44:47.757963144 (769h15m47.629046179s) [upload=673036]: Next returned nil segment
Jan  6 16:44:47.757975688 (769h15m47.629058723s) [upload=673036]: Waiting for error group managing segments...
Jan  6 16:44:48.175355824 (769h15m48.046438859s) [noise=true , address=145.79.227.36:4529   , piece_id=YNZEOHMFBHOMAIZBV7FPRTBXSA3THOQ2R5XEND4RRGL4PNLM2W5A, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.189871995 (769h15m48.06095503s) [noise=true , address=145.79.227.3:4160    , piece_id=MVK7OHYWBQL3U6ICJFFUCLGDMIA436VRAVYHDNZRX6TZ7KME25RA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.207607777 (769h15m48.078690822s) [noise=true , address=64.49.13.138:4486    , piece_id=TDFOABDUDWDEMPDFFUIR6J7H6OCYFETTBJHCCIPGJUO5NMIJEGUA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.227981511 (769h15m48.099064556s) [noise=true , address=145.79.227.23:4573   , piece_id=DAKGS66HRC7OAAGDMI7XU4GBYT4MYRWBOFCH7RF2A4KP3VUIQA2Q, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.247897795 (769h15m48.11898084s) [noise=true , address=145.79.227.20:4455   , piece_id=ZHAXGWI47TGWTZA4XU6XSZTD3GRCSG6AUU7N5S7VQXTAJAVDIDJQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.349641861 (769h15m48.220724906s) [noise=true , address=145.79.227.13:4808   , piece_id=UEMCSOO4WUMC2VQYHTKX7DH6XZMAD44HOTTUYMID34AZXJEDGXDA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.782145845 (769h15m48.65322888s) [noise=true , address=145.79.227.4:4198    , piece_id=E6RB3OBCCHSFBRIHHHR56E7M5RZ23TNYYQPPQ5UKE4RP2MW42J3Q, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:48.997160406 (769h15m48.868243451s) [noise=true , address=45.91.116.12:4235    , piece_id=C3GJTOVEHMK6CQ3TWC7NGO4CXMGCQ4VC4WO6EMN646TVFZLFTSCQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.027918673 (769h15m48.899001708s) [noise=true , address=193.149.164.143:4695 , piece_id=Y5VL4E75Q5ULJCK4CLWBR43WOFXRO226UTHHH4JABUIVOT2TFJOA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.084408278 (769h15m48.955491313s) [noise=true , address=77.90.31.12:4601     , piece_id=BLD52KPVFQQLCK2MVZ2EHJHLRMIVK47DNUE7JRTGPXBD4R4WSFWQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.128152033 (769h15m48.999235079s) [noise=true , address=145.79.227.29:4971   , piece_id=ROIKGGPDHXZ2XCSVIAP4ETUD6ACOKIIVG7EVYCYO6AL47G7G6ZIQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.133368196 (769h15m49.004451231s) [noise=true , address=145.79.227.44:4216   , piece_id=2PBU2YR4KAMTVSVEG4NRLL6IZPQMX3DJCBXVG3K72LZQMLDQWQSA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.134414158 (769h15m49.005497203s) [noise=true , address=145.79.227.40:4053   , piece_id=LUTH3OGSOTO6OYSCTHVIDAT57QDG32FYGXBTMQCAR5F3PXJU5OMQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.212147304 (769h15m49.083230349s) [noise=true , address=145.79.227.49:4362   , piece_id=OD3PZGZU2NBXSFGEJYVWADPJHGSC33ERBTKGRSVDNKGPC66YX77A, seg_pos={0 0}, upload=673036]: Done uploading piece. err: <nil>
Jan  6 16:44:49.212173132 (769h15m49.083256178s) [seg_pos={0 0}, upload=673036]: Segment reached optimal threshold of 14 pieces.
Jan  6 16:44:49.212410895 (769h15m49.08349393s) [noise=true , address=145.79.227.45:4250   , piece_id=HPEJNGEZKKZTQGFDMREMI2HT2YFOWIH2XYKCI525UHGOIHZPD7VA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oHdA7jYDKSYwWLB975bJ3Fu27xjfZDC9utpnQpszmYjVz2CdD): protocol: write tcp 188.166.225.170:58078->145.79.227.45:4250: use of closed network connection; write tcp 188.166.225.170:58078->145.79.227.45:4250: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58078->145.79.227.45:4250: use of closed network connection
Jan  6 16:44:49.212419001 (769h15m49.083502036s) [noise=true , address=145.79.227.45:4250   , piece_id=HPEJNGEZKKZTQGFDMREMI2HT2YFOWIH2XYKCI525UHGOIHZPD7VA, seg_pos={0 0}, upload=673036]: Long tail cancellation due to optimal threshold
Jan  6 16:44:49.212661392 (769h15m49.083744427s) [noise=true , address=45.91.116.8:4053     , piece_id=WKZWJ4EBW2DR5JIROXLBNFUNBEAMGDYZSKHIAAT4YD6F4YSDW3SQ, seg_pos={0 0}, upload=673036]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LA2misujs8mqKZRnYqV4kCZoG46KZaxjyYGqT1HdpzEFhyBf6): protocol: read tcp 188.166.225.170:37938->45.91.116.8:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan  6 16:44:49.212690538 (769h15m49.083773583s) [noise=true , address=45.91.116.8:4053     , piece_id=WKZWJ4EBW2DR5JIROXLBNFUNBEAMGDYZSKHIAAT4YD6F4YSDW3SQ, seg_pos={0 0}, upload=673036]: Long tail cancellation due to optimal threshold
Jan  6 16:44:49.212707120 (769h15m49.083790155s) [noise=true , address=145.79.227.7:4302    , piece_id=EDW7TST2H5EDOW2R3KTYI7IZGWUFEVJ3YA7KBBZZ42ESO56LGBGA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288AZgzLsbgYuXzMt5exDkjAyUUokGHFRD8dK9f88uDep7oVxH): protocol: write tcp 188.166.225.170:52152->145.79.227.7:4302: use of closed network connection; write tcp 188.166.225.170:52152->145.79.227.7:4302: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52152->145.79.227.7:4302: use of closed network connection
Jan  6 16:44:49.212715395 (769h15m49.083798441s) [noise=true , address=145.79.227.7:4302    , piece_id=EDW7TST2H5EDOW2R3KTYI7IZGWUFEVJ3YA7KBBZZ42ESO56LGBGA, seg_pos={0 0}, upload=673036]: Long tail cancellation due to optimal threshold
Jan  6 16:44:49.212904215 (769h15m49.08398726s) [noise=true , address=145.79.227.33:4894   , piece_id=D7SPQ5SNK5SPFQWZE54LAJWG3TRKV4NUOBGANXMF2YB37ASIAANA, seg_pos={0 0}, upload=673036]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF
Jan  6 16:44:49.212923271 (769h15m49.084006306s) [noise=true , address=145.79.227.33:4894   , piece_id=D7SPQ5SNK5SPFQWZE54LAJWG3TRKV4NUOBGANXMF2YB37ASIAANA, seg_pos={0 0}, upload=673036]: Long tail cancellation due to optimal threshold
Jan  6 16:44:49.212943820 (769h15m49.084026865s) [seg_pos={0 0}, upload=673036]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jan  6 16:44:49.213046986 (769h15m49.084130031s) [upload=673036]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:44:49.213051696 (769h15m49.084134731s) [upload=673036]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:44:49.517610893 (769h15m49.388693949s) [upload=673036]: Done uploading segments...
Jan  6 16:44:49.517643054 (769h15m49.388726089s) [upload=673036]: Upload finished. err: <nil>
Duration: 1m0.005923655s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
57 out of 60 is failed (threshold=6) [XXXXXXXX.XX.XXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
5.000694444s
trace:
https://jaeger.dev.storj.io/trace/785b1f8c6499123
FAIL (2026-01-06 16:45:38 UTC)
orbiter warning: context deadline exceeded
Duration: 5.000712428s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)
antiblip:
2 out of 60 is failed (threshold=3) [..............................................XX............]
duration:
15.412423547s
trace:
https://jaeger.dev.storj.io/trace/4b3b841c440bfc62
FAIL (2026-01-06 16:45:35 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
JXEZNOOX75ENDV6EBXU4NR4HW4GTZMVA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913312874 (769h5m40.784395909s) [noise=true , address=81.171.14.70:4035    , piece_id=BRXIJD7JX7L37MWMTVPL5KSESQUQEZNRXLBLL26XDMYIJSCRUD5A, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913698277 (769h5m40.784781322s) [noise=true , address=81.171.14.70:4030    , piece_id=WV5BULPUQSAE7DUO2FWY74KF3MIHSQXYC673GNJ7XUBQXXM6KHEQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913862180 (769h5m40.784945215s) [noise=true , address=89.149.201.147:4004  , piece_id=NMPCYNSE3YVIAVDCWRO4VP6IN4DKZ2P4YIFYI6ZDTEU75QSFC4NQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.914346662 (769h5m40.785429707s) [noise=true , address=185.167.204.205:4035 , piece_id=HPHYG44QNKJYFSXEBH6FPYLFXLO6OUIVGDYL5GXJMP3PDNFUZEQA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.914842456 (769h5m40.785925491s) [noise=true , address=81.171.14.70:4034    , piece_id=5LGDXPCHMOQPZGJAEH3ZSPF64RSTVAJRQFZBXVWP5D3Y36YFLUUQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.915307461 (769h5m40.786390506s) [noise=true , address=89.149.201.147:4019  , piece_id=LS3NXHELDI7H6VCTHO6LAGRDDG46ST4IF2XOFZDZ6EK756K5ACHQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.915793817 (769h5m40.786876863s) [noise=true , address=185.167.204.206:4067 , piece_id=GLITDKTEKYV4CFGL3TJXWJDFOBYPFX2UTNJFPUCZDJM3HOWVFS4Q, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.916279973 (769h5m40.787363018s) [noise=true , address=89.149.201.147:4014  , piece_id=KNV6OXYOC2H7LVZCW4ZWQRYRN5ZCMII3OO3FAS3YYQ22574KUCRQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.916759336 (769h5m40.787842371s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.920089677 (769h5m40.791172722s) [noise=true , address=185.167.204.205:4014 , piece_id=XUCDQV4VW4ZZFTSZV2YEDAO5GQFP2ASW6BJFFC735UWEVIR3RNXA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:41.741185400 (769h5m41.612268445s) [upload=672894]: Next returned nil segment
Jan  6 16:34:41.741201020 (769h5m41.612284055s) [upload=672894]: Waiting for error group managing segments...
Jan  6 16:34:42.136024704 (769h5m42.00710774s) [noise=true , address=185.167.204.205:4036 , piece_id=4XKAIZPEJZAVAS7P2QILKY7XIDIIRIRWH7HTN5OVXUHAL2Y3H4SQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.137413950 (769h5m42.008496995s) [noise=true , address=185.167.204.206:4051 , piece_id=XWKJM6ZIHISLSESJ2ZQQQMZTGMK5MHH6L4ZD5RXJ3SMAVASVVVOA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.137527506 (769h5m42.008610541s) [noise=true , address=185.167.204.206:4049 , piece_id=4T665GS7XTBVEHUVT6SFIQW4F3TTBPDKZ57PKQHHHWWKC4P4JQPQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.138183485 (769h5m42.00926653s) [noise=true , address=185.167.204.206:4048 , piece_id=ZGQYUYRK5PK24UF4PYT67FHNPR4GVFDCJ2OEGRFENLB2GYR2LFVQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.139902609 (769h5m42.010985644s) [noise=true , address=185.167.204.206:4056 , piece_id=KYHDW5RLJTZCURXCFIP7HN3ZYLPPHK7FTO6IIGQ5VJJUPF4ICQXA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.140277843 (769h5m42.011360878s) [noise=true , address=185.167.204.206:4067 , piece_id=GLITDKTEKYV4CFGL3TJXWJDFOBYPFX2UTNJFPUCZDJM3HOWVFS4Q, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.142394303 (769h5m42.013477348s) [noise=true , address=185.167.204.205:4021 , piece_id=U4X3NZRT7MHG2JTNRVDRYIC72UDMHUQUCY4VFW7JFVLLVACZN7OA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.179529514 (769h5m42.050612549s) [noise=true , address=185.167.204.206:4053 , piece_id=HBFCBJCOLPJLNE4INHP4JXEZNOOX75ENDV6EBXU4NR4HW4GTZMVA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.194387988 (769h5m42.065471023s) [noise=true , address=185.167.204.205:4035 , piece_id=HPHYG44QNKJYFSXEBH6FPYLFXLO6OUIVGDYL5GXJMP3PDNFUZEQA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.197860059 (769h5m42.068943094s) [noise=true , address=185.167.204.206:4064 , piece_id=JGCXE3QUORVKLRIWEXFZUCT4XUF3DRVA35ZKQU4DQODLZBOH5I4A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.827615690 (769h5m42.698698725s) [noise=true , address=81.171.14.70:4041    , piece_id=NLGMNQJDFISSAHBX3EC6WMWJLP74FMR4MRPUIX4GCCCQRG5RRRCQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.917676689 (769h5m42.788759725s) [noise=true , address=81.171.14.70:4034    , piece_id=5LGDXPCHMOQPZGJAEH3ZSPF64RSTVAJRQFZBXVWP5D3Y36YFLUUQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.932213350 (769h5m42.803296385s) [noise=true , address=81.171.14.70:4030    , piece_id=WV5BULPUQSAE7DUO2FWY74KF3MIHSQXYC673GNJ7XUBQXXM6KHEQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.956051121 (769h5m42.827134196s) [noise=true , address=81.171.14.70:4031    , piece_id=OJKJHNVWTPDITZC5CTQNB7UFPIVZDPE54ZXS4LWRM3XHYAB6443A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.005259852 (769h5m42.876342887s) [noise=true , address=185.167.204.205:4019 , piece_id=I7OP7AQS2DC3C7GZWG5RVJSLDKAZD2434WMBPRYMZFVAJYJG3JHQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.052708792 (769h5m42.923791837s) [noise=true , address=81.171.14.70:4046    , piece_id=OV3PMCD6UIFNSQNTCOS3GEQC4676KFJOUH5ZX576OFOYC5EXCZPQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.639749858 (769h5m43.510832903s) [noise=true , address=89.149.201.147:4004  , piece_id=NMPCYNSE3YVIAVDCWRO4VP6IN4DKZ2P4YIFYI6ZDTEU75QSFC4NQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.719270598 (769h5m43.590353643s) [noise=true , address=89.149.201.147:4019  , piece_id=LS3NXHELDI7H6VCTHO6LAGRDDG46ST4IF2XOFZDZ6EK756K5ACHQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.892052420 (769h5m43.763135466s) [noise=true , address=81.171.14.70:4035    , piece_id=BRXIJD7JX7L37MWMTVPL5KSESQUQEZNRXLBLL26XDMYIJSCRUD5A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.901787498 (769h5m43.772870533s) [noise=true , address=81.171.14.70:4039    , piece_id=VFXOSS5DRL43RDJGPLSH3667HWKAF5ZNC2CF5BA7TDR4NSCG75HA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.019887208 (769h5m43.890970243s) [noise=true , address=185.167.204.206:4050 , piece_id=KSCVNGXUGFV7XRNU5MFIKLLQGZ5BNHTIXQLVHWPMITZOQUA23DGQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.510337921 (769h5m44.381420957s) [noise=true , address=185.167.204.205:4028 , piece_id=QPB4EPPEENW277S4SBIOCWWMBLIE6GKCGEM3TV3MXN6SYFMQAI7Q, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.692198242 (769h5m44.563281277s) [noise=true , address=89.149.201.147:4010  , piece_id=C57ORVBIZE4QUWCVG3OVG3KW2Z2GYVWUXEOYG5R53KV2WUQLR7JQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.394574949 (769h5m46.265657984s) [noise=true , address=89.149.201.147:4014  , piece_id=KNV6OXYOC2H7LVZCW4ZWQRYRN5ZCMII3OO3FAS3YYQ22574KUCRQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.543864889 (769h5m46.414947934s) [noise=true , address=185.167.204.205:4014 , piece_id=XUCDQV4VW4ZZFTSZV2YEDAO5GQFP2ASW6BJFFC735UWEVIR3RNXA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.846238946 (769h5m46.717321982s) [noise=true , address=185.167.204.205:4006 , piece_id=W3XXEI7MG37VJHHK2CECNKIIHB7WSQABGVB5W5ZGAWSTFQ4NQMEQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.846258273 (769h5m46.717341308s) [seg_pos={0 0}, upload=672894]: Segment reached optimal threshold of 26 pieces.
Jan  6 16:34:46.846448986 (769h5m46.717532021s) [noise=true , address=185.167.204.206:4072 , piece_id=EJ4XCLNDZETIFIKE3FK2Q4NA35KT5LOTNGE253MKUCAIGH4FZD5A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v): protocol: expected piece hash; context canceled; EOF
Jan  6 16:34:46.846454998 (769h5m46.717538033s) [noise=true , address=185.167.204.206:4072 , piece_id=EJ4XCLNDZETIFIKE3FK2Q4NA35KT5LOTNGE253MKUCAIGH4FZD5A, seg_pos={0 0}, upload=672894]: Long tail cancellation due to optimal threshold
Jan  6 16:34:46.846579175 (769h5m46.71766221s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK): protocol: expected piece hash; context canceled; EOF
Jan  6 16:34:46.846595475 (769h5m46.71767851s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Long tail cancellation due to optimal threshold
Jan  6 16:34:46.846620453 (769h5m46.717703488s) [seg_pos={0 0}, upload=672894]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan  6 16:34:46.846729380 (769h5m46.717812415s) [upload=672894]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:34:46.846734600 (769h5m46.717817635s) [upload=672894]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:34:47.079572108 (769h5m46.950655143s) [upload=672894]: Done uploading segments...
Jan  6 16:34:47.079590854 (769h5m46.950673889s) [upload=672894]: Upload finished. err: <nil>
Duration: 15.412451819s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
11 out of 60 is failed (threshold=3) [....................X.X....X.X.X....X.X...X....XXX..........]
duration:
16.305614818s
trace:
https://jaeger.dev.storj.io/trace/73dd0c7413e4b545
FAIL (2026-01-06 16:45:31 UTC)
orbiter warning: context deadline exceeded
Duration: 16.305642691s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
22 out of 60 is failed (threshold=3) [XXXX..X.....XXX........X.......X..XX.....XX.X...X.XXX.XX..X.]
duration:
5.762036115s
trace:
https://jaeger.dev.storj.io/trace/7508a6819d5b778c
FAIL (2026-01-06 16:45:24 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, Piece ID: B3ENUUJCJ3GWSDWJ3K2EJ4P6LMWUZ2SREHOQJ3EXEKKYRRGDUTKQ) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: Z3HIXJDXNQA5VWI6UVKWP7TR6I2BUVWCKQLQTZ5HPT7TSS73GD6Q) context deadline exceeded
======LOGS======
Jan  6 16:44:48.993245861 (769h15m48.864328916s) [upload=673038]: Starting upload
Jan  6 16:44:48.993321757 (769h15m48.864404792s) [upload=673038]: Done starting upload
Jan  6 16:44:48.993445612 (769h15m48.864528667s) [upload=673038]: Uploading segments...
Jan  6 16:44:48.993673566 (769h15m48.864756611s) [upload=673038]: Got next segment. Inline: false
Jan  6 16:44:48.993717309 (769h15m48.864800375s) [upload=673038]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan  6 16:44:48.993723050 (769h15m48.864806085s) [upload=673038]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan  6 16:44:49.244178736 (769h15m49.115261771s) [seg_pos={0 0}, upload=673038]: Begin upload segment...
Jan  6 16:44:49.244224253 (769h15m49.115307298s) [seg_pos={0 0}, upload=673038]: Done begin upload segment.
Jan  6 16:44:49.244249130 (769h15m49.115332165s) [noise=true , address=45.91.116.4:4940     , piece_id=HZR2HIJYTV67JGEWAEJWQXRRPMXUORCB4HSSODZUTTG4RCF7PRFQ, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.244347488 (769h15m49.115430533s) [noise=true , address=145.79.227.41:4085   , piece_id=SXAX26RX7FCCDYHZ4FWYZN3ZZ222JV7E3RZGLHXRRLRRSYE6BM3A, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.244898167 (769h15m49.115981202s) [noise=true , address=145.79.227.27:4690   , piece_id=Q3ODYVT2UOZP4FXPMVCM23QOXE7BQMYXWAL2L743RTM3TYMW6OVA, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.245352872 (769h15m49.116435907s) [noise=true , address=45.91.116.6:4975     , piece_id=B3ENUUJCJ3GWSDWJ3K2EJ4P6LMWUZ2SREHOQJ3EXEKKYRRGDUTKQ, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.245669215 (769h15m49.11675226s) [noise=true , address=145.79.227.16:4912   , piece_id=MVWP7ZHJ3D7F6MTZNSRFLTPCYYI4J6G32ROFPPNT2SPBNSKGTBZA, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.246055260 (769h15m49.117138295s) [noise=true , address=145.79.227.33:4869   , piece_id=2ZXGCF4U537JPCCBZUDBNHIPQ3U34L3DUGSZQZV7MQJ5EYFQNSSQ, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.246373826 (769h15m49.117456861s) [noise=true , address=45.91.116.9:4262     , piece_id=QOWGX5LYAYPIYSGLVGJS4CLNXT445LHXUSZTXCIC7MEEQ6WCKFJA, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.246425294 (769h15m49.117508329s) [noise=true , address=145.79.227.26:4655   , piece_id=QM2QZW4OGBLUUOADRJNYQ62VQ45GCM5JHFCKDY6ZH6722Y66ZGBA, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.246696170 (769h15m49.117779215s) [noise=true , address=45.91.116.7:4012     , piece_id=Z3HIXJDXNQA5VWI6UVKWP7TR6I2BUVWCKQLQTZ5HPT7TSS73GD6Q, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.247026038 (769h15m49.118109073s) [noise=true , address=145.79.227.34:4907   , piece_id=FDV4PLX7NS5IXTDFS4HPVIBFE42XPHJRGSAWRHD4QXCCEMJFV4IA, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.247142209 (769h15m49.118225254s) [noise=true , address=145.79.227.15:4874   , piece_id=M4EI33GN3CXY42HCYQWRWS7FXSFOJVTVZ75UJMFATYGJJUR4XC3A, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.247545528 (769h15m49.118628573s) [noise=true , address=145.79.227.45:4231   , piece_id=6LVMSB6Z2TTINMFM77KY62U5N7TI3GURKXQXQHBBXSN3C5DY24WQ, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.247571426 (769h15m49.118654471s) [noise=true , address=145.79.227.21:4475   , piece_id=GVEY3SQKJIS3SEAQ5W5JUSNL4ZZLI6S77BMYVECZUH3T7OGWUP3A, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.247880024 (769h15m49.118963059s) [noise=true , address=145.79.227.5:4223    , piece_id=ATJZUKHQO5GJFCUK643RKCH7VORA26GGJSGGMVWR2TMQ54ZJWL7A, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.248331043 (769h15m49.119414088s) [noise=true , address=45.91.116.11:4118    , piece_id=RSYDERQN4NRQH3YBG3ME2PWT3XP4EIGN77DOPJ6PMCZEHVHLYMDQ, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.248818942 (769h15m49.119901987s) [noise=true , address=45.91.116.10:4081    , piece_id=TKD4EGCWC7ECQBHH3J3VOFWOMIGWDO6SKEUVIK5VTEXYPDYIDZLQ, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:50.947532052 (769h15m50.818615097s) [upload=673038]: Next returned nil segment
Jan  6 16:44:50.947550929 (769h15m50.818633974s) [upload=673038]: Waiting for error group managing segments...
Jan  6 16:44:51.230732389 (769h15m51.101815444s) [noise=true , address=145.79.227.5:4223    , piece_id=ATJZUKHQO5GJFCUK643RKCH7VORA26GGJSGGMVWR2TMQ54ZJWL7A, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:51.243612895 (769h15m51.11469593s) [noise=true , address=145.79.227.45:4231   , piece_id=6LVMSB6Z2TTINMFM77KY62U5N7TI3GURKXQXQHBBXSN3C5DY24WQ, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:51.245500330 (769h15m51.116583365s) [noise=true , address=145.79.227.16:4912   , piece_id=MVWP7ZHJ3D7F6MTZNSRFLTPCYYI4J6G32ROFPPNT2SPBNSKGTBZA, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:51.250551869 (769h15m51.121634904s) [noise=true , address=145.79.227.15:4874   , piece_id=M4EI33GN3CXY42HCYQWRWS7FXSFOJVTVZ75UJMFATYGJJUR4XC3A, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:51.251449829 (769h15m51.122532864s) [noise=true , address=145.79.227.27:4690   , piece_id=Q3ODYVT2UOZP4FXPMVCM23QOXE7BQMYXWAL2L743RTM3TYMW6OVA, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:51.257483256 (769h15m51.128566301s) [noise=true , address=145.79.227.26:4655   , piece_id=QM2QZW4OGBLUUOADRJNYQ62VQ45GCM5JHFCKDY6ZH6722Y66ZGBA, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:51.313489671 (769h15m51.184572706s) [noise=true , address=145.79.227.41:4085   , piece_id=SXAX26RX7FCCDYHZ4FWYZN3ZZ222JV7E3RZGLHXRRLRRSYE6BM3A, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:51.315148360 (769h15m51.186231405s) [noise=true , address=145.79.227.21:4475   , piece_id=GVEY3SQKJIS3SEAQ5W5JUSNL4ZZLI6S77BMYVECZUH3T7OGWUP3A, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:51.678092731 (769h15m51.549175766s) [noise=true , address=45.91.116.4:4940     , piece_id=HZR2HIJYTV67JGEWAEJWQXRRPMXUORCB4HSSODZUTTG4RCF7PRFQ, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:52.104467674 (769h15m51.97555071s) [noise=true , address=45.91.116.10:4081    , piece_id=TKD4EGCWC7ECQBHH3J3VOFWOMIGWDO6SKEUVIK5VTEXYPDYIDZLQ, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:52.376790137 (769h15m52.247873183s) [noise=true , address=45.91.116.6:4975     , piece_id=B3ENUUJCJ3GWSDWJ3K2EJ4P6LMWUZ2SREHOQJ3EXEKKYRRGDUTKQ, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:52.420398184 (769h15m52.291481229s) [noise=true , address=45.91.116.9:4262     , piece_id=QOWGX5LYAYPIYSGLVGJS4CLNXT445LHXUSZTXCIC7MEEQ6WCKFJA, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:52.531293435 (769h15m52.40237647s) [noise=true , address=45.91.116.7:4012     , piece_id=Z3HIXJDXNQA5VWI6UVKWP7TR6I2BUVWCKQLQTZ5HPT7TSS73GD6Q, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:52.531313073 (769h15m52.402396108s) [seg_pos={0 0}, upload=673038]: Segment reached optimal threshold of 13 pieces.
Jan  6 16:44:52.531514386 (769h15m52.402597422s) [noise=true , address=45.91.116.11:4118    , piece_id=RSYDERQN4NRQH3YBG3ME2PWT3XP4EIGN77DOPJ6PMCZEHVHLYMDQ, seg_pos={0 0}, upload=673038]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jan  6 16:44:52.531520488 (769h15m52.402603523s) [noise=true , address=45.91.116.11:4118    , piece_id=RSYDERQN4NRQH3YBG3ME2PWT3XP4EIGN77DOPJ6PMCZEHVHLYMDQ, seg_pos={0 0}, upload=673038]: Long tail cancellation due to optimal threshold
Jan  6 16:44:52.531709557 (769h15m52.402792593s) [noise=true , address=145.79.227.34:4907   , piece_id=FDV4PLX7NS5IXTDFS4HPVIBFE42XPHJRGSAWRHD4QXCCEMJFV4IA, seg_pos={0 0}, upload=673038]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Jan  6 16:44:52.531715408 (769h15m52.402798444s) [noise=true , address=145.79.227.34:4907   , piece_id=FDV4PLX7NS5IXTDFS4HPVIBFE42XPHJRGSAWRHD4QXCCEMJFV4IA, seg_pos={0 0}, upload=673038]: Long tail cancellation due to optimal threshold
Jan  6 16:44:52.531743682 (769h15m52.402826738s) [noise=true , address=145.79.227.33:4869   , piece_id=2ZXGCF4U537JPCCBZUDBNHIPQ3U34L3DUGSZQZV7MQJ5EYFQNSSQ, seg_pos={0 0}, upload=673038]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CQxjq65r6xVTmNRurSQPzKxj4BoR6cJ8Uq29sTrdh6uXm5sq): protocol: expected piece hash; context canceled; EOF
Jan  6 16:44:52.531762599 (769h15m52.402845634s) [noise=true , address=145.79.227.33:4869   , piece_id=2ZXGCF4U537JPCCBZUDBNHIPQ3U34L3DUGSZQZV7MQJ5EYFQNSSQ, seg_pos={0 0}, upload=673038]: Long tail cancellation due to optimal threshold
Jan  6 16:44:52.531792867 (769h15m52.402875902s) [seg_pos={0 0}, upload=673038]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan  6 16:44:52.531903607 (769h15m52.402986652s) [upload=673038]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:44:52.531910631 (769h15m52.402993676s) [upload=673038]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:44:52.871731993 (769h15m52.742815028s) [upload=673038]: Done uploading segments...
Jan  6 16:44:52.871750979 (769h15m52.742834024s) [upload=673038]: Upload finished. err: <nil>
Duration: 5.762073836s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
2 out of 60 is failed (threshold=3) [....................X......X................................]
duration:
17.362926807s
trace:
https://jaeger.dev.storj.io/trace/d5e2d8b8127d900
FAIL (2026-01-06 16:45:20 UTC)
orbiter warning: context deadline exceeded
Duration: 17.362946564s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
22 out of 60 is failed (threshold=3) [.XXXX..X.....XXX........X.......X..XX.....XX.X...X.XXX.XX..X]
duration:
30.00352844s
trace:
https://jaeger.dev.storj.io/trace/7508a6819d5b778c
FAIL (2026-01-06 16:45:18 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, Piece ID: B3ENUUJCJ3GWSDWJ3K2EJ4P6LMWUZ2SREHOQJ3EXEKKYRRGDUTKQ) context deadline exceeded; piecestore: (Node ID: 1smg5DiC8TRGnnmoQrRpvScvehv8aqtA2DaKtD1JtAhrhnm96t, Piece ID: Z3HIXJDXNQA5VWI6UVKWP7TR6I2BUVWCKQLQTZ5HPT7TSS73GD6Q) context deadline exceeded
======LOGS======
Jan  6 16:44:48.993245861 (769h15m48.864328916s) [upload=673038]: Starting upload
Jan  6 16:44:48.993321757 (769h15m48.864404792s) [upload=673038]: Done starting upload
Jan  6 16:44:48.993445612 (769h15m48.864528667s) [upload=673038]: Uploading segments...
Jan  6 16:44:48.993673566 (769h15m48.864756611s) [upload=673038]: Got next segment. Inline: false
Jan  6 16:44:48.993717309 (769h15m48.864800375s) [upload=673038]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan  6 16:44:48.993723050 (769h15m48.864806085s) [upload=673038]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan  6 16:44:49.244178736 (769h15m49.115261771s) [seg_pos={0 0}, upload=673038]: Begin upload segment...
Jan  6 16:44:49.244224253 (769h15m49.115307298s) [seg_pos={0 0}, upload=673038]: Done begin upload segment.
Jan  6 16:44:49.244249130 (769h15m49.115332165s) [noise=true , address=45.91.116.4:4940     , piece_id=HZR2HIJYTV67JGEWAEJWQXRRPMXUORCB4HSSODZUTTG4RCF7PRFQ, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.244347488 (769h15m49.115430533s) [noise=true , address=145.79.227.41:4085   , piece_id=SXAX26RX7FCCDYHZ4FWYZN3ZZ222JV7E3RZGLHXRRLRRSYE6BM3A, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.244898167 (769h15m49.115981202s) [noise=true , address=145.79.227.27:4690   , piece_id=Q3ODYVT2UOZP4FXPMVCM23QOXE7BQMYXWAL2L743RTM3TYMW6OVA, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.245352872 (769h15m49.116435907s) [noise=true , address=45.91.116.6:4975     , piece_id=B3ENUUJCJ3GWSDWJ3K2EJ4P6LMWUZ2SREHOQJ3EXEKKYRRGDUTKQ, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.245669215 (769h15m49.11675226s) [noise=true , address=145.79.227.16:4912   , piece_id=MVWP7ZHJ3D7F6MTZNSRFLTPCYYI4J6G32ROFPPNT2SPBNSKGTBZA, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.246055260 (769h15m49.117138295s) [noise=true , address=145.79.227.33:4869   , piece_id=2ZXGCF4U537JPCCBZUDBNHIPQ3U34L3DUGSZQZV7MQJ5EYFQNSSQ, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.246373826 (769h15m49.117456861s) [noise=true , address=45.91.116.9:4262     , piece_id=QOWGX5LYAYPIYSGLVGJS4CLNXT445LHXUSZTXCIC7MEEQ6WCKFJA, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.246425294 (769h15m49.117508329s) [noise=true , address=145.79.227.26:4655   , piece_id=QM2QZW4OGBLUUOADRJNYQ62VQ45GCM5JHFCKDY6ZH6722Y66ZGBA, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.246696170 (769h15m49.117779215s) [noise=true , address=45.91.116.7:4012     , piece_id=Z3HIXJDXNQA5VWI6UVKWP7TR6I2BUVWCKQLQTZ5HPT7TSS73GD6Q, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.247026038 (769h15m49.118109073s) [noise=true , address=145.79.227.34:4907   , piece_id=FDV4PLX7NS5IXTDFS4HPVIBFE42XPHJRGSAWRHD4QXCCEMJFV4IA, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.247142209 (769h15m49.118225254s) [noise=true , address=145.79.227.15:4874   , piece_id=M4EI33GN3CXY42HCYQWRWS7FXSFOJVTVZ75UJMFATYGJJUR4XC3A, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.247545528 (769h15m49.118628573s) [noise=true , address=145.79.227.45:4231   , piece_id=6LVMSB6Z2TTINMFM77KY62U5N7TI3GURKXQXQHBBXSN3C5DY24WQ, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.247571426 (769h15m49.118654471s) [noise=true , address=145.79.227.21:4475   , piece_id=GVEY3SQKJIS3SEAQ5W5JUSNL4ZZLI6S77BMYVECZUH3T7OGWUP3A, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.247880024 (769h15m49.118963059s) [noise=true , address=145.79.227.5:4223    , piece_id=ATJZUKHQO5GJFCUK643RKCH7VORA26GGJSGGMVWR2TMQ54ZJWL7A, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.248331043 (769h15m49.119414088s) [noise=true , address=45.91.116.11:4118    , piece_id=RSYDERQN4NRQH3YBG3ME2PWT3XP4EIGN77DOPJ6PMCZEHVHLYMDQ, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:49.248818942 (769h15m49.119901987s) [noise=true , address=45.91.116.10:4081    , piece_id=TKD4EGCWC7ECQBHH3J3VOFWOMIGWDO6SKEUVIK5VTEXYPDYIDZLQ, seg_pos={0 0}, upload=673038]: Uploading piece...
Jan  6 16:44:50.947532052 (769h15m50.818615097s) [upload=673038]: Next returned nil segment
Jan  6 16:44:50.947550929 (769h15m50.818633974s) [upload=673038]: Waiting for error group managing segments...
Jan  6 16:44:51.230732389 (769h15m51.101815444s) [noise=true , address=145.79.227.5:4223    , piece_id=ATJZUKHQO5GJFCUK643RKCH7VORA26GGJSGGMVWR2TMQ54ZJWL7A, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:51.243612895 (769h15m51.11469593s) [noise=true , address=145.79.227.45:4231   , piece_id=6LVMSB6Z2TTINMFM77KY62U5N7TI3GURKXQXQHBBXSN3C5DY24WQ, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:51.245500330 (769h15m51.116583365s) [noise=true , address=145.79.227.16:4912   , piece_id=MVWP7ZHJ3D7F6MTZNSRFLTPCYYI4J6G32ROFPPNT2SPBNSKGTBZA, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:51.250551869 (769h15m51.121634904s) [noise=true , address=145.79.227.15:4874   , piece_id=M4EI33GN3CXY42HCYQWRWS7FXSFOJVTVZ75UJMFATYGJJUR4XC3A, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:51.251449829 (769h15m51.122532864s) [noise=true , address=145.79.227.27:4690   , piece_id=Q3ODYVT2UOZP4FXPMVCM23QOXE7BQMYXWAL2L743RTM3TYMW6OVA, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:51.257483256 (769h15m51.128566301s) [noise=true , address=145.79.227.26:4655   , piece_id=QM2QZW4OGBLUUOADRJNYQ62VQ45GCM5JHFCKDY6ZH6722Y66ZGBA, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:51.313489671 (769h15m51.184572706s) [noise=true , address=145.79.227.41:4085   , piece_id=SXAX26RX7FCCDYHZ4FWYZN3ZZ222JV7E3RZGLHXRRLRRSYE6BM3A, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:51.315148360 (769h15m51.186231405s) [noise=true , address=145.79.227.21:4475   , piece_id=GVEY3SQKJIS3SEAQ5W5JUSNL4ZZLI6S77BMYVECZUH3T7OGWUP3A, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:51.678092731 (769h15m51.549175766s) [noise=true , address=45.91.116.4:4940     , piece_id=HZR2HIJYTV67JGEWAEJWQXRRPMXUORCB4HSSODZUTTG4RCF7PRFQ, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:52.104467674 (769h15m51.97555071s) [noise=true , address=45.91.116.10:4081    , piece_id=TKD4EGCWC7ECQBHH3J3VOFWOMIGWDO6SKEUVIK5VTEXYPDYIDZLQ, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:52.376790137 (769h15m52.247873183s) [noise=true , address=45.91.116.6:4975     , piece_id=B3ENUUJCJ3GWSDWJ3K2EJ4P6LMWUZ2SREHOQJ3EXEKKYRRGDUTKQ, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:52.420398184 (769h15m52.291481229s) [noise=true , address=45.91.116.9:4262     , piece_id=QOWGX5LYAYPIYSGLVGJS4CLNXT445LHXUSZTXCIC7MEEQ6WCKFJA, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:52.531293435 (769h15m52.40237647s) [noise=true , address=45.91.116.7:4012     , piece_id=Z3HIXJDXNQA5VWI6UVKWP7TR6I2BUVWCKQLQTZ5HPT7TSS73GD6Q, seg_pos={0 0}, upload=673038]: Done uploading piece. err: <nil>
Jan  6 16:44:52.531313073 (769h15m52.402396108s) [seg_pos={0 0}, upload=673038]: Segment reached optimal threshold of 13 pieces.
Jan  6 16:44:52.531514386 (769h15m52.402597422s) [noise=true , address=45.91.116.11:4118    , piece_id=RSYDERQN4NRQH3YBG3ME2PWT3XP4EIGN77DOPJ6PMCZEHVHLYMDQ, seg_pos={0 0}, upload=673038]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF
Jan  6 16:44:52.531520488 (769h15m52.402603523s) [noise=true , address=45.91.116.11:4118    , piece_id=RSYDERQN4NRQH3YBG3ME2PWT3XP4EIGN77DOPJ6PMCZEHVHLYMDQ, seg_pos={0 0}, upload=673038]: Long tail cancellation due to optimal threshold
Jan  6 16:44:52.531709557 (769h15m52.402792593s) [noise=true , address=145.79.227.34:4907   , piece_id=FDV4PLX7NS5IXTDFS4HPVIBFE42XPHJRGSAWRHD4QXCCEMJFV4IA, seg_pos={0 0}, upload=673038]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12URx18Ckh7VaAKJ1hdQQucMRu5qiDrAraqaycjYqxETgwZ4uWS): protocol: expected piece hash; context canceled; EOF
Jan  6 16:44:52.531715408 (769h15m52.402798444s) [noise=true , address=145.79.227.34:4907   , piece_id=FDV4PLX7NS5IXTDFS4HPVIBFE42XPHJRGSAWRHD4QXCCEMJFV4IA, seg_pos={0 0}, upload=673038]: Long tail cancellation due to optimal threshold
Jan  6 16:44:52.531743682 (769h15m52.402826738s) [noise=true , address=145.79.227.33:4869   , piece_id=2ZXGCF4U537JPCCBZUDBNHIPQ3U34L3DUGSZQZV7MQJ5EYFQNSSQ, seg_pos={0 0}, upload=673038]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18CQxjq65r6xVTmNRurSQPzKxj4BoR6cJ8Uq29sTrdh6uXm5sq): protocol: expected piece hash; context canceled; EOF
Jan  6 16:44:52.531762599 (769h15m52.402845634s) [noise=true , address=145.79.227.33:4869   , piece_id=2ZXGCF4U537JPCCBZUDBNHIPQ3U34L3DUGSZQZV7MQJ5EYFQNSSQ, seg_pos={0 0}, upload=673038]: Long tail cancellation due to optimal threshold
Jan  6 16:44:52.531792867 (769h15m52.402875902s) [seg_pos={0 0}, upload=673038]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan  6 16:44:52.531903607 (769h15m52.402986652s) [upload=673038]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:44:52.531910631 (769h15m52.402993676s) [upload=673038]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:44:52.871731993 (769h15m52.742815028s) [upload=673038]: Done uploading segments...
Jan  6 16:44:52.871750979 (769h15m52.742834024s) [upload=673038]: Upload finished. err: <nil>
Duration: 30.003555431s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
57 out of 60 is failed (threshold=6) [XXXXXXXXX.XX.XXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
5.001309698s
trace:
https://jaeger.dev.storj.io/trace/21790bcf18aeaff3
FAIL (2026-01-06 16:44:58 UTC)
orbiter warning: context deadline exceeded
Duration: 5.001365716s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)
antiblip:
2 out of 60 is failed (threshold=3) [...............................................XX...........]
duration:
15.343672456s
trace:
https://jaeger.dev.storj.io/trace/4b3b841c440bfc62
FAIL (2026-01-06 16:44:48 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
JXEZNOOX75ENDV6EBXU4NR4HW4GTZMVA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913312874 (769h5m40.784395909s) [noise=true , address=81.171.14.70:4035    , piece_id=BRXIJD7JX7L37MWMTVPL5KSESQUQEZNRXLBLL26XDMYIJSCRUD5A, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913698277 (769h5m40.784781322s) [noise=true , address=81.171.14.70:4030    , piece_id=WV5BULPUQSAE7DUO2FWY74KF3MIHSQXYC673GNJ7XUBQXXM6KHEQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913862180 (769h5m40.784945215s) [noise=true , address=89.149.201.147:4004  , piece_id=NMPCYNSE3YVIAVDCWRO4VP6IN4DKZ2P4YIFYI6ZDTEU75QSFC4NQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.914346662 (769h5m40.785429707s) [noise=true , address=185.167.204.205:4035 , piece_id=HPHYG44QNKJYFSXEBH6FPYLFXLO6OUIVGDYL5GXJMP3PDNFUZEQA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.914842456 (769h5m40.785925491s) [noise=true , address=81.171.14.70:4034    , piece_id=5LGDXPCHMOQPZGJAEH3ZSPF64RSTVAJRQFZBXVWP5D3Y36YFLUUQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.915307461 (769h5m40.786390506s) [noise=true , address=89.149.201.147:4019  , piece_id=LS3NXHELDI7H6VCTHO6LAGRDDG46ST4IF2XOFZDZ6EK756K5ACHQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.915793817 (769h5m40.786876863s) [noise=true , address=185.167.204.206:4067 , piece_id=GLITDKTEKYV4CFGL3TJXWJDFOBYPFX2UTNJFPUCZDJM3HOWVFS4Q, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.916279973 (769h5m40.787363018s) [noise=true , address=89.149.201.147:4014  , piece_id=KNV6OXYOC2H7LVZCW4ZWQRYRN5ZCMII3OO3FAS3YYQ22574KUCRQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.916759336 (769h5m40.787842371s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.920089677 (769h5m40.791172722s) [noise=true , address=185.167.204.205:4014 , piece_id=XUCDQV4VW4ZZFTSZV2YEDAO5GQFP2ASW6BJFFC735UWEVIR3RNXA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:41.741185400 (769h5m41.612268445s) [upload=672894]: Next returned nil segment
Jan  6 16:34:41.741201020 (769h5m41.612284055s) [upload=672894]: Waiting for error group managing segments...
Jan  6 16:34:42.136024704 (769h5m42.00710774s) [noise=true , address=185.167.204.205:4036 , piece_id=4XKAIZPEJZAVAS7P2QILKY7XIDIIRIRWH7HTN5OVXUHAL2Y3H4SQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.137413950 (769h5m42.008496995s) [noise=true , address=185.167.204.206:4051 , piece_id=XWKJM6ZIHISLSESJ2ZQQQMZTGMK5MHH6L4ZD5RXJ3SMAVASVVVOA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.137527506 (769h5m42.008610541s) [noise=true , address=185.167.204.206:4049 , piece_id=4T665GS7XTBVEHUVT6SFIQW4F3TTBPDKZ57PKQHHHWWKC4P4JQPQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.138183485 (769h5m42.00926653s) [noise=true , address=185.167.204.206:4048 , piece_id=ZGQYUYRK5PK24UF4PYT67FHNPR4GVFDCJ2OEGRFENLB2GYR2LFVQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.139902609 (769h5m42.010985644s) [noise=true , address=185.167.204.206:4056 , piece_id=KYHDW5RLJTZCURXCFIP7HN3ZYLPPHK7FTO6IIGQ5VJJUPF4ICQXA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.140277843 (769h5m42.011360878s) [noise=true , address=185.167.204.206:4067 , piece_id=GLITDKTEKYV4CFGL3TJXWJDFOBYPFX2UTNJFPUCZDJM3HOWVFS4Q, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.142394303 (769h5m42.013477348s) [noise=true , address=185.167.204.205:4021 , piece_id=U4X3NZRT7MHG2JTNRVDRYIC72UDMHUQUCY4VFW7JFVLLVACZN7OA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.179529514 (769h5m42.050612549s) [noise=true , address=185.167.204.206:4053 , piece_id=HBFCBJCOLPJLNE4INHP4JXEZNOOX75ENDV6EBXU4NR4HW4GTZMVA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.194387988 (769h5m42.065471023s) [noise=true , address=185.167.204.205:4035 , piece_id=HPHYG44QNKJYFSXEBH6FPYLFXLO6OUIVGDYL5GXJMP3PDNFUZEQA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.197860059 (769h5m42.068943094s) [noise=true , address=185.167.204.206:4064 , piece_id=JGCXE3QUORVKLRIWEXFZUCT4XUF3DRVA35ZKQU4DQODLZBOH5I4A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.827615690 (769h5m42.698698725s) [noise=true , address=81.171.14.70:4041    , piece_id=NLGMNQJDFISSAHBX3EC6WMWJLP74FMR4MRPUIX4GCCCQRG5RRRCQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.917676689 (769h5m42.788759725s) [noise=true , address=81.171.14.70:4034    , piece_id=5LGDXPCHMOQPZGJAEH3ZSPF64RSTVAJRQFZBXVWP5D3Y36YFLUUQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.932213350 (769h5m42.803296385s) [noise=true , address=81.171.14.70:4030    , piece_id=WV5BULPUQSAE7DUO2FWY74KF3MIHSQXYC673GNJ7XUBQXXM6KHEQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.956051121 (769h5m42.827134196s) [noise=true , address=81.171.14.70:4031    , piece_id=OJKJHNVWTPDITZC5CTQNB7UFPIVZDPE54ZXS4LWRM3XHYAB6443A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.005259852 (769h5m42.876342887s) [noise=true , address=185.167.204.205:4019 , piece_id=I7OP7AQS2DC3C7GZWG5RVJSLDKAZD2434WMBPRYMZFVAJYJG3JHQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.052708792 (769h5m42.923791837s) [noise=true , address=81.171.14.70:4046    , piece_id=OV3PMCD6UIFNSQNTCOS3GEQC4676KFJOUH5ZX576OFOYC5EXCZPQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.639749858 (769h5m43.510832903s) [noise=true , address=89.149.201.147:4004  , piece_id=NMPCYNSE3YVIAVDCWRO4VP6IN4DKZ2P4YIFYI6ZDTEU75QSFC4NQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.719270598 (769h5m43.590353643s) [noise=true , address=89.149.201.147:4019  , piece_id=LS3NXHELDI7H6VCTHO6LAGRDDG46ST4IF2XOFZDZ6EK756K5ACHQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.892052420 (769h5m43.763135466s) [noise=true , address=81.171.14.70:4035    , piece_id=BRXIJD7JX7L37MWMTVPL5KSESQUQEZNRXLBLL26XDMYIJSCRUD5A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.901787498 (769h5m43.772870533s) [noise=true , address=81.171.14.70:4039    , piece_id=VFXOSS5DRL43RDJGPLSH3667HWKAF5ZNC2CF5BA7TDR4NSCG75HA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.019887208 (769h5m43.890970243s) [noise=true , address=185.167.204.206:4050 , piece_id=KSCVNGXUGFV7XRNU5MFIKLLQGZ5BNHTIXQLVHWPMITZOQUA23DGQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.510337921 (769h5m44.381420957s) [noise=true , address=185.167.204.205:4028 , piece_id=QPB4EPPEENW277S4SBIOCWWMBLIE6GKCGEM3TV3MXN6SYFMQAI7Q, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.692198242 (769h5m44.563281277s) [noise=true , address=89.149.201.147:4010  , piece_id=C57ORVBIZE4QUWCVG3OVG3KW2Z2GYVWUXEOYG5R53KV2WUQLR7JQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.394574949 (769h5m46.265657984s) [noise=true , address=89.149.201.147:4014  , piece_id=KNV6OXYOC2H7LVZCW4ZWQRYRN5ZCMII3OO3FAS3YYQ22574KUCRQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.543864889 (769h5m46.414947934s) [noise=true , address=185.167.204.205:4014 , piece_id=XUCDQV4VW4ZZFTSZV2YEDAO5GQFP2ASW6BJFFC735UWEVIR3RNXA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.846238946 (769h5m46.717321982s) [noise=true , address=185.167.204.205:4006 , piece_id=W3XXEI7MG37VJHHK2CECNKIIHB7WSQABGVB5W5ZGAWSTFQ4NQMEQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.846258273 (769h5m46.717341308s) [seg_pos={0 0}, upload=672894]: Segment reached optimal threshold of 26 pieces.
Jan  6 16:34:46.846448986 (769h5m46.717532021s) [noise=true , address=185.167.204.206:4072 , piece_id=EJ4XCLNDZETIFIKE3FK2Q4NA35KT5LOTNGE253MKUCAIGH4FZD5A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v): protocol: expected piece hash; context canceled; EOF
Jan  6 16:34:46.846454998 (769h5m46.717538033s) [noise=true , address=185.167.204.206:4072 , piece_id=EJ4XCLNDZETIFIKE3FK2Q4NA35KT5LOTNGE253MKUCAIGH4FZD5A, seg_pos={0 0}, upload=672894]: Long tail cancellation due to optimal threshold
Jan  6 16:34:46.846579175 (769h5m46.71766221s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK): protocol: expected piece hash; context canceled; EOF
Jan  6 16:34:46.846595475 (769h5m46.71767851s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Long tail cancellation due to optimal threshold
Jan  6 16:34:46.846620453 (769h5m46.717703488s) [seg_pos={0 0}, upload=672894]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan  6 16:34:46.846729380 (769h5m46.717812415s) [upload=672894]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:34:46.846734600 (769h5m46.717817635s) [upload=672894]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:34:47.079572108 (769h5m46.950655143s) [upload=672894]: Done uploading segments...
Jan  6 16:34:47.079590854 (769h5m46.950673889s) [upload=672894]: Upload finished. err: <nil>
Duration: 15.34369574s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
11 out of 60 is failed (threshold=3) [.....................X.X....X.X.X....X.X...X....XXX.........]
duration:
7.599772317s
trace:
https://jaeger.dev.storj.io/trace/73dd0c7413e4b545
FAIL (2026-01-06 16:44:24 UTC)
orbiter warning: context deadline exceeded
Duration: 7.599823235s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
2 out of 60 is failed (threshold=3) [.....................X......X...............................]
duration:
6.129398189s
trace:
https://jaeger.dev.storj.io/trace/d5e2d8b8127d900
FAIL (2026-01-06 16:44:22 UTC)
orbiter warning: context deadline exceeded
Duration: 6.129423197s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)
antiblip:
2 out of 60 is failed (threshold=3) [................................................XX..........]
duration:
13.725991625s
trace:
https://jaeger.dev.storj.io/trace/4b3b841c440bfc62
FAIL (2026-01-06 16:44:19 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
JXEZNOOX75ENDV6EBXU4NR4HW4GTZMVA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913312874 (769h5m40.784395909s) [noise=true , address=81.171.14.70:4035    , piece_id=BRXIJD7JX7L37MWMTVPL5KSESQUQEZNRXLBLL26XDMYIJSCRUD5A, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913698277 (769h5m40.784781322s) [noise=true , address=81.171.14.70:4030    , piece_id=WV5BULPUQSAE7DUO2FWY74KF3MIHSQXYC673GNJ7XUBQXXM6KHEQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913862180 (769h5m40.784945215s) [noise=true , address=89.149.201.147:4004  , piece_id=NMPCYNSE3YVIAVDCWRO4VP6IN4DKZ2P4YIFYI6ZDTEU75QSFC4NQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.914346662 (769h5m40.785429707s) [noise=true , address=185.167.204.205:4035 , piece_id=HPHYG44QNKJYFSXEBH6FPYLFXLO6OUIVGDYL5GXJMP3PDNFUZEQA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.914842456 (769h5m40.785925491s) [noise=true , address=81.171.14.70:4034    , piece_id=5LGDXPCHMOQPZGJAEH3ZSPF64RSTVAJRQFZBXVWP5D3Y36YFLUUQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.915307461 (769h5m40.786390506s) [noise=true , address=89.149.201.147:4019  , piece_id=LS3NXHELDI7H6VCTHO6LAGRDDG46ST4IF2XOFZDZ6EK756K5ACHQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.915793817 (769h5m40.786876863s) [noise=true , address=185.167.204.206:4067 , piece_id=GLITDKTEKYV4CFGL3TJXWJDFOBYPFX2UTNJFPUCZDJM3HOWVFS4Q, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.916279973 (769h5m40.787363018s) [noise=true , address=89.149.201.147:4014  , piece_id=KNV6OXYOC2H7LVZCW4ZWQRYRN5ZCMII3OO3FAS3YYQ22574KUCRQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.916759336 (769h5m40.787842371s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.920089677 (769h5m40.791172722s) [noise=true , address=185.167.204.205:4014 , piece_id=XUCDQV4VW4ZZFTSZV2YEDAO5GQFP2ASW6BJFFC735UWEVIR3RNXA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:41.741185400 (769h5m41.612268445s) [upload=672894]: Next returned nil segment
Jan  6 16:34:41.741201020 (769h5m41.612284055s) [upload=672894]: Waiting for error group managing segments...
Jan  6 16:34:42.136024704 (769h5m42.00710774s) [noise=true , address=185.167.204.205:4036 , piece_id=4XKAIZPEJZAVAS7P2QILKY7XIDIIRIRWH7HTN5OVXUHAL2Y3H4SQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.137413950 (769h5m42.008496995s) [noise=true , address=185.167.204.206:4051 , piece_id=XWKJM6ZIHISLSESJ2ZQQQMZTGMK5MHH6L4ZD5RXJ3SMAVASVVVOA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.137527506 (769h5m42.008610541s) [noise=true , address=185.167.204.206:4049 , piece_id=4T665GS7XTBVEHUVT6SFIQW4F3TTBPDKZ57PKQHHHWWKC4P4JQPQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.138183485 (769h5m42.00926653s) [noise=true , address=185.167.204.206:4048 , piece_id=ZGQYUYRK5PK24UF4PYT67FHNPR4GVFDCJ2OEGRFENLB2GYR2LFVQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.139902609 (769h5m42.010985644s) [noise=true , address=185.167.204.206:4056 , piece_id=KYHDW5RLJTZCURXCFIP7HN3ZYLPPHK7FTO6IIGQ5VJJUPF4ICQXA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.140277843 (769h5m42.011360878s) [noise=true , address=185.167.204.206:4067 , piece_id=GLITDKTEKYV4CFGL3TJXWJDFOBYPFX2UTNJFPUCZDJM3HOWVFS4Q, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.142394303 (769h5m42.013477348s) [noise=true , address=185.167.204.205:4021 , piece_id=U4X3NZRT7MHG2JTNRVDRYIC72UDMHUQUCY4VFW7JFVLLVACZN7OA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.179529514 (769h5m42.050612549s) [noise=true , address=185.167.204.206:4053 , piece_id=HBFCBJCOLPJLNE4INHP4JXEZNOOX75ENDV6EBXU4NR4HW4GTZMVA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.194387988 (769h5m42.065471023s) [noise=true , address=185.167.204.205:4035 , piece_id=HPHYG44QNKJYFSXEBH6FPYLFXLO6OUIVGDYL5GXJMP3PDNFUZEQA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.197860059 (769h5m42.068943094s) [noise=true , address=185.167.204.206:4064 , piece_id=JGCXE3QUORVKLRIWEXFZUCT4XUF3DRVA35ZKQU4DQODLZBOH5I4A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.827615690 (769h5m42.698698725s) [noise=true , address=81.171.14.70:4041    , piece_id=NLGMNQJDFISSAHBX3EC6WMWJLP74FMR4MRPUIX4GCCCQRG5RRRCQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.917676689 (769h5m42.788759725s) [noise=true , address=81.171.14.70:4034    , piece_id=5LGDXPCHMOQPZGJAEH3ZSPF64RSTVAJRQFZBXVWP5D3Y36YFLUUQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.932213350 (769h5m42.803296385s) [noise=true , address=81.171.14.70:4030    , piece_id=WV5BULPUQSAE7DUO2FWY74KF3MIHSQXYC673GNJ7XUBQXXM6KHEQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.956051121 (769h5m42.827134196s) [noise=true , address=81.171.14.70:4031    , piece_id=OJKJHNVWTPDITZC5CTQNB7UFPIVZDPE54ZXS4LWRM3XHYAB6443A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.005259852 (769h5m42.876342887s) [noise=true , address=185.167.204.205:4019 , piece_id=I7OP7AQS2DC3C7GZWG5RVJSLDKAZD2434WMBPRYMZFVAJYJG3JHQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.052708792 (769h5m42.923791837s) [noise=true , address=81.171.14.70:4046    , piece_id=OV3PMCD6UIFNSQNTCOS3GEQC4676KFJOUH5ZX576OFOYC5EXCZPQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.639749858 (769h5m43.510832903s) [noise=true , address=89.149.201.147:4004  , piece_id=NMPCYNSE3YVIAVDCWRO4VP6IN4DKZ2P4YIFYI6ZDTEU75QSFC4NQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.719270598 (769h5m43.590353643s) [noise=true , address=89.149.201.147:4019  , piece_id=LS3NXHELDI7H6VCTHO6LAGRDDG46ST4IF2XOFZDZ6EK756K5ACHQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.892052420 (769h5m43.763135466s) [noise=true , address=81.171.14.70:4035    , piece_id=BRXIJD7JX7L37MWMTVPL5KSESQUQEZNRXLBLL26XDMYIJSCRUD5A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.901787498 (769h5m43.772870533s) [noise=true , address=81.171.14.70:4039    , piece_id=VFXOSS5DRL43RDJGPLSH3667HWKAF5ZNC2CF5BA7TDR4NSCG75HA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.019887208 (769h5m43.890970243s) [noise=true , address=185.167.204.206:4050 , piece_id=KSCVNGXUGFV7XRNU5MFIKLLQGZ5BNHTIXQLVHWPMITZOQUA23DGQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.510337921 (769h5m44.381420957s) [noise=true , address=185.167.204.205:4028 , piece_id=QPB4EPPEENW277S4SBIOCWWMBLIE6GKCGEM3TV3MXN6SYFMQAI7Q, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.692198242 (769h5m44.563281277s) [noise=true , address=89.149.201.147:4010  , piece_id=C57ORVBIZE4QUWCVG3OVG3KW2Z2GYVWUXEOYG5R53KV2WUQLR7JQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.394574949 (769h5m46.265657984s) [noise=true , address=89.149.201.147:4014  , piece_id=KNV6OXYOC2H7LVZCW4ZWQRYRN5ZCMII3OO3FAS3YYQ22574KUCRQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.543864889 (769h5m46.414947934s) [noise=true , address=185.167.204.205:4014 , piece_id=XUCDQV4VW4ZZFTSZV2YEDAO5GQFP2ASW6BJFFC735UWEVIR3RNXA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.846238946 (769h5m46.717321982s) [noise=true , address=185.167.204.205:4006 , piece_id=W3XXEI7MG37VJHHK2CECNKIIHB7WSQABGVB5W5ZGAWSTFQ4NQMEQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.846258273 (769h5m46.717341308s) [seg_pos={0 0}, upload=672894]: Segment reached optimal threshold of 26 pieces.
Jan  6 16:34:46.846448986 (769h5m46.717532021s) [noise=true , address=185.167.204.206:4072 , piece_id=EJ4XCLNDZETIFIKE3FK2Q4NA35KT5LOTNGE253MKUCAIGH4FZD5A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v): protocol: expected piece hash; context canceled; EOF
Jan  6 16:34:46.846454998 (769h5m46.717538033s) [noise=true , address=185.167.204.206:4072 , piece_id=EJ4XCLNDZETIFIKE3FK2Q4NA35KT5LOTNGE253MKUCAIGH4FZD5A, seg_pos={0 0}, upload=672894]: Long tail cancellation due to optimal threshold
Jan  6 16:34:46.846579175 (769h5m46.71766221s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK): protocol: expected piece hash; context canceled; EOF
Jan  6 16:34:46.846595475 (769h5m46.71767851s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Long tail cancellation due to optimal threshold
Jan  6 16:34:46.846620453 (769h5m46.717703488s) [seg_pos={0 0}, upload=672894]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan  6 16:34:46.846729380 (769h5m46.717812415s) [upload=672894]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:34:46.846734600 (769h5m46.717817635s) [upload=672894]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:34:47.079572108 (769h5m46.950655143s) [upload=672894]: Done uploading segments...
Jan  6 16:34:47.079590854 (769h5m46.950673889s) [upload=672894]: Upload finished. err: <nil>
Duration: 13.726022373s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
12 out of 60 is failed (threshold=3) [.....X..X.X.X........X.....XX......X..........XX...X.X......]
duration:
15.495405109s
trace:
https://jaeger.dev.storj.io/trace/76df23b0035fe317
FAIL (2026-01-06 16:44:15 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
g piece...
Jan  6 16:39:13.319055742 (769h10m13.190138787s) [noise=true , address=77.90.31.12:4616     , piece_id=JCWWA7PGIW23J4OKRJYW33S37FGDTIHBHTCLJSXXA4RH4SE7SWUA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.319538962 (769h10m13.190621997s) [noise=true , address=45.91.116.7:4014     , piece_id=65DKIVFKLQVVJEHZOT2P4KYDBHFD6URVGOFPFPABEGTOORENHXBA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.320135148 (769h10m13.191218193s) [noise=true , address=77.90.31.14:4668     , piece_id=QYCFWBXHW7BYLILL76W5MDFYQJKG6VZIZOYS3BF4OPDZLMJZXKVQ, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.320651351 (769h10m13.191734396s) [noise=true , address=145.79.227.11:4736   , piece_id=CTSU3X6OZJFWJSCLJBJ3EBGALWWGZL62ZJN436LKICHA4MLIQ2ZA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.321172283 (769h10m13.192255328s) [noise=true , address=64.49.13.27:4968     , piece_id=X4RAE7BUCM2R5NDMCCLH4TRGRCCXJBGUXZRE4QSNCFWNCJXLBSVA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.323456752 (769h10m13.194539787s) [noise=true , address=145.79.227.12:4773   , piece_id=JXMIF6EZKPD7EXZ646Z4XM6PG6I5P3EPTBJUWYQX4LMGAUFJT4EA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.325413879 (769h10m13.196496924s) [noise=true , address=145.79.227.33:4884   , piece_id=E5YFUCC75VTQBLZH3T5Q6VWEMRYAVOXNFKZQCPEP3ES64YCTXWPA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.327557260 (769h10m13.198640305s) [noise=true , address=145.79.227.6:4279    , piece_id=FNXBVE7QJDR67LLYLBXOCK3OQ7E6O5BIKKXPQMYOSMKIZSNK2EDQ, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.329566295 (769h10m13.20064934s) [noise=true , address=45.91.116.9:4282     , piece_id=GC72TML4SY7YTFQW4G3CVTOMGW5HWQLCWHBFQ3TKSZSUJUR252WA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.330791288 (769h10m13.201874323s) [noise=true , address=45.91.116.11:4139    , piece_id=NRCGKCNT7VWR4UUJE7HLXWT23K3BRUV2AHMDPOAVAXQF4UZKIPUA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.332349425 (769h10m13.20343246s) [noise=true , address=145.79.227.49:4371   , piece_id=YNFO36VNDUUPFQO4OFHIUEWUZWLYM2BUD7NF6VQEVWZHNCDSM6FA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.332771599 (769h10m13.203854634s) [noise=true , address=145.79.227.23:4557   , piece_id=QHDDTZ4DZAJCIU3I5HVA54JQLOHCCN73GSJAWZGC5JL6X4WV7ETA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.333853448 (769h10m13.204936494s) [noise=true , address=145.79.227.14:4843   , piece_id=GSANAVILLP6UTITEPDCBK5VHTTK7D2SBT7FQQI77H35I3SLNRVKA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.336038750 (769h10m13.207121785s) [noise=true , address=45.91.116.10:4107    , piece_id=I5OP5FXQBURDQG2LWEKGVZUBX7VV3CLD6Z2ZKKDK5UCX2RGFX3BA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:14.793189300 (769h10m14.664272335s) [upload=672957]: Next returned nil segment
Jan  6 16:39:14.793201272 (769h10m14.664284307s) [upload=672957]: Waiting for error group managing segments...
Jan  6 16:39:15.209058789 (769h10m15.080141834s) [noise=true , address=145.79.227.13:4835   , piece_id=MMTT3F5VEI2NUZ4T2QWJGNCPIB3O4EGLYANWMCFK6MC3YLQX3TXA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.251874898 (769h10m15.122957933s) [noise=true , address=145.79.227.14:4843   , piece_id=GSANAVILLP6UTITEPDCBK5VHTTK7D2SBT7FQQI77H35I3SLNRVKA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.299718689 (769h10m15.170801724s) [noise=true , address=145.79.227.11:4736   , piece_id=CTSU3X6OZJFWJSCLJBJ3EBGALWWGZL62ZJN436LKICHA4MLIQ2ZA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.351276473 (769h10m15.222359518s) [noise=true , address=145.79.227.12:4773   , piece_id=JXMIF6EZKPD7EXZ646Z4XM6PG6I5P3EPTBJUWYQX4LMGAUFJT4EA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.423449492 (769h10m15.294532527s) [noise=true , address=64.49.13.27:4968     , piece_id=X4RAE7BUCM2R5NDMCCLH4TRGRCCXJBGUXZRE4QSNCFWNCJXLBSVA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.580777038 (769h10m15.451860083s) [noise=true , address=77.90.31.14:4668     , piece_id=QYCFWBXHW7BYLILL76W5MDFYQJKG6VZIZOYS3BF4OPDZLMJZXKVQ, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.798468394 (769h10m15.66955143s) [noise=true , address=45.91.116.10:4107    , piece_id=I5OP5FXQBURDQG2LWEKGVZUBX7VV3CLD6Z2ZKKDK5UCX2RGFX3BA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.083877398 (769h10m15.954960443s) [noise=true , address=45.91.116.9:4282     , piece_id=GC72TML4SY7YTFQW4G3CVTOMGW5HWQLCWHBFQ3TKSZSUJUR252WA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.143050622 (769h10m16.014133687s) [noise=true , address=145.79.227.32:4865   , piece_id=Z5ZJEDUJG376CVJM4EDK6JPRJK2U5OR6RDD4YG5X4MXH4WKSZYSA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.194307403 (769h10m16.065390438s) [noise=true , address=145.79.227.33:4884   , piece_id=E5YFUCC75VTQBLZH3T5Q6VWEMRYAVOXNFKZQCPEP3ES64YCTXWPA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.244019653 (769h10m16.115102688s) [noise=true , address=145.79.227.49:4371   , piece_id=YNFO36VNDUUPFQO4OFHIUEWUZWLYM2BUD7NF6VQEVWZHNCDSM6FA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.587195683 (769h10m16.458278718s) [noise=true , address=45.91.116.7:4014     , piece_id=65DKIVFKLQVVJEHZOT2P4KYDBHFD6URVGOFPFPABEGTOORENHXBA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.752240113 (769h10m16.623323148s) [noise=true , address=145.79.227.21:4501   , piece_id=KQIHUVS3N56R5HFH2S2I6OE7PBZUXAHRGFMPK57KX7UA5I7BLCSQ, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.752279798 (769h10m16.623362843s) [noise=true , address=77.90.31.12:4616     , piece_id=JCWWA7PGIW23J4OKRJYW33S37FGDTIHBHTCLJSXXA4RH4SE7SWUA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.752291660 (769h10m16.623374695s) [seg_pos={0 0}, upload=672957]: Segment reached optimal threshold of 14 pieces.
Jan  6 16:39:16.752502462 (769h10m16.623585507s) [noise=true , address=145.79.227.6:4279    , piece_id=FNXBVE7QJDR67LLYLBXOCK3OQ7E6O5BIKKXPQMYOSMKIZSNK2EDQ, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uiA6tmZtmHDNaNtkhRtH31q4MJxn3C4bfrLAXmYz5mKmofHoG): protocol: write tcp 188.166.225.170:35010->145.79.227.6:4279: use of closed network connection; write tcp 188.166.225.170:35010->145.79.227.6:4279: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35010->145.79.227.6:4279: use of closed network connection
Jan  6 16:39:16.752510658 (769h10m16.623593693s) [noise=true , address=145.79.227.6:4279    , piece_id=FNXBVE7QJDR67LLYLBXOCK3OQ7E6O5BIKKXPQMYOSMKIZSNK2EDQ, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752633512 (769h10m16.623716557s) [noise=true , address=145.79.227.20:4454   , piece_id=W7E7EUDKE6BTN5I7TSTAMHETYKCXBU7RLYALUHELLOSTOAUFVB2Q, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SafkJPqKHLwMaqyZdLq3yJ1xdjqDtWHFrUwynmegUVwEC7y5J): protocol: expected piece hash; context canceled; EOF
Jan  6 16:39:16.752641587 (769h10m16.623724632s) [noise=true , address=145.79.227.20:4454   , piece_id=W7E7EUDKE6BTN5I7TSTAMHETYKCXBU7RLYALUHELLOSTOAUFVB2Q, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752764281 (769h10m16.623847316s) [noise=true , address=145.79.227.23:4557   , piece_id=QHDDTZ4DZAJCIU3I5HVA54JQLOHCCN73GSJAWZGC5JL6X4WV7ETA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sF7TCu1pL8mrtjoPULJ2VnZQHcJSMmvou2w7YgF58ZHQcnfJT): protocol: write tcp 188.166.225.170:43038->145.79.227.23:4557: use of closed network connection; write tcp 188.166.225.170:43038->145.79.227.23:4557: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43038->145.79.227.23:4557: use of closed network connection
Jan  6 16:39:16.752774249 (769h10m16.623857295s) [noise=true , address=145.79.227.23:4557   , piece_id=QHDDTZ4DZAJCIU3I5HVA54JQLOHCCN73GSJAWZGC5JL6X4WV7ETA, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752797775 (769h10m16.62388081s) [noise=true , address=45.91.116.11:4139    , piece_id=NRCGKCNT7VWR4UUJE7HLXWT23K3BRUV2AHMDPOAVAXQF4UZKIPUA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Tz3HEHACmHU9Gp2W2vHVuF5PXNmdsa5N8nYhwt52HCTjggqRD): protocol: read tcp 188.166.225.170:44870->45.91.116.11:4139: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan  6 16:39:16.752805439 (769h10m16.623888484s) [noise=true , address=45.91.116.11:4139    , piece_id=NRCGKCNT7VWR4UUJE7HLXWT23K3BRUV2AHMDPOAVAXQF4UZKIPUA, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752825017 (769h10m16.623908052s) [seg_pos={0 0}, upload=672957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jan  6 16:39:16.752902604 (769h10m16.623985659s) [upload=672957]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:39:16.752918193 (769h10m16.624001229s) [upload=672957]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:39:17.044336649 (769h10m16.915419684s) [upload=672957]: Done uploading segments...
Jan  6 16:39:17.044357138 (769h10m16.915440173s) [upload=672957]: Upload finished. err: <nil>
Duration: 15.495438101s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
12 out of 60 is failed (threshold=3) [......X..X.X.X........X.....XX......X..........XX...X.X.....]
duration:
15.556088329s
trace:
https://jaeger.dev.storj.io/trace/76df23b0035fe317
FAIL (2026-01-06 16:43:54 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
g piece...
Jan  6 16:39:13.319055742 (769h10m13.190138787s) [noise=true , address=77.90.31.12:4616     , piece_id=JCWWA7PGIW23J4OKRJYW33S37FGDTIHBHTCLJSXXA4RH4SE7SWUA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.319538962 (769h10m13.190621997s) [noise=true , address=45.91.116.7:4014     , piece_id=65DKIVFKLQVVJEHZOT2P4KYDBHFD6URVGOFPFPABEGTOORENHXBA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.320135148 (769h10m13.191218193s) [noise=true , address=77.90.31.14:4668     , piece_id=QYCFWBXHW7BYLILL76W5MDFYQJKG6VZIZOYS3BF4OPDZLMJZXKVQ, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.320651351 (769h10m13.191734396s) [noise=true , address=145.79.227.11:4736   , piece_id=CTSU3X6OZJFWJSCLJBJ3EBGALWWGZL62ZJN436LKICHA4MLIQ2ZA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.321172283 (769h10m13.192255328s) [noise=true , address=64.49.13.27:4968     , piece_id=X4RAE7BUCM2R5NDMCCLH4TRGRCCXJBGUXZRE4QSNCFWNCJXLBSVA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.323456752 (769h10m13.194539787s) [noise=true , address=145.79.227.12:4773   , piece_id=JXMIF6EZKPD7EXZ646Z4XM6PG6I5P3EPTBJUWYQX4LMGAUFJT4EA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.325413879 (769h10m13.196496924s) [noise=true , address=145.79.227.33:4884   , piece_id=E5YFUCC75VTQBLZH3T5Q6VWEMRYAVOXNFKZQCPEP3ES64YCTXWPA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.327557260 (769h10m13.198640305s) [noise=true , address=145.79.227.6:4279    , piece_id=FNXBVE7QJDR67LLYLBXOCK3OQ7E6O5BIKKXPQMYOSMKIZSNK2EDQ, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.329566295 (769h10m13.20064934s) [noise=true , address=45.91.116.9:4282     , piece_id=GC72TML4SY7YTFQW4G3CVTOMGW5HWQLCWHBFQ3TKSZSUJUR252WA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.330791288 (769h10m13.201874323s) [noise=true , address=45.91.116.11:4139    , piece_id=NRCGKCNT7VWR4UUJE7HLXWT23K3BRUV2AHMDPOAVAXQF4UZKIPUA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.332349425 (769h10m13.20343246s) [noise=true , address=145.79.227.49:4371   , piece_id=YNFO36VNDUUPFQO4OFHIUEWUZWLYM2BUD7NF6VQEVWZHNCDSM6FA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.332771599 (769h10m13.203854634s) [noise=true , address=145.79.227.23:4557   , piece_id=QHDDTZ4DZAJCIU3I5HVA54JQLOHCCN73GSJAWZGC5JL6X4WV7ETA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.333853448 (769h10m13.204936494s) [noise=true , address=145.79.227.14:4843   , piece_id=GSANAVILLP6UTITEPDCBK5VHTTK7D2SBT7FQQI77H35I3SLNRVKA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.336038750 (769h10m13.207121785s) [noise=true , address=45.91.116.10:4107    , piece_id=I5OP5FXQBURDQG2LWEKGVZUBX7VV3CLD6Z2ZKKDK5UCX2RGFX3BA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:14.793189300 (769h10m14.664272335s) [upload=672957]: Next returned nil segment
Jan  6 16:39:14.793201272 (769h10m14.664284307s) [upload=672957]: Waiting for error group managing segments...
Jan  6 16:39:15.209058789 (769h10m15.080141834s) [noise=true , address=145.79.227.13:4835   , piece_id=MMTT3F5VEI2NUZ4T2QWJGNCPIB3O4EGLYANWMCFK6MC3YLQX3TXA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.251874898 (769h10m15.122957933s) [noise=true , address=145.79.227.14:4843   , piece_id=GSANAVILLP6UTITEPDCBK5VHTTK7D2SBT7FQQI77H35I3SLNRVKA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.299718689 (769h10m15.170801724s) [noise=true , address=145.79.227.11:4736   , piece_id=CTSU3X6OZJFWJSCLJBJ3EBGALWWGZL62ZJN436LKICHA4MLIQ2ZA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.351276473 (769h10m15.222359518s) [noise=true , address=145.79.227.12:4773   , piece_id=JXMIF6EZKPD7EXZ646Z4XM6PG6I5P3EPTBJUWYQX4LMGAUFJT4EA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.423449492 (769h10m15.294532527s) [noise=true , address=64.49.13.27:4968     , piece_id=X4RAE7BUCM2R5NDMCCLH4TRGRCCXJBGUXZRE4QSNCFWNCJXLBSVA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.580777038 (769h10m15.451860083s) [noise=true , address=77.90.31.14:4668     , piece_id=QYCFWBXHW7BYLILL76W5MDFYQJKG6VZIZOYS3BF4OPDZLMJZXKVQ, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.798468394 (769h10m15.66955143s) [noise=true , address=45.91.116.10:4107    , piece_id=I5OP5FXQBURDQG2LWEKGVZUBX7VV3CLD6Z2ZKKDK5UCX2RGFX3BA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.083877398 (769h10m15.954960443s) [noise=true , address=45.91.116.9:4282     , piece_id=GC72TML4SY7YTFQW4G3CVTOMGW5HWQLCWHBFQ3TKSZSUJUR252WA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.143050622 (769h10m16.014133687s) [noise=true , address=145.79.227.32:4865   , piece_id=Z5ZJEDUJG376CVJM4EDK6JPRJK2U5OR6RDD4YG5X4MXH4WKSZYSA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.194307403 (769h10m16.065390438s) [noise=true , address=145.79.227.33:4884   , piece_id=E5YFUCC75VTQBLZH3T5Q6VWEMRYAVOXNFKZQCPEP3ES64YCTXWPA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.244019653 (769h10m16.115102688s) [noise=true , address=145.79.227.49:4371   , piece_id=YNFO36VNDUUPFQO4OFHIUEWUZWLYM2BUD7NF6VQEVWZHNCDSM6FA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.587195683 (769h10m16.458278718s) [noise=true , address=45.91.116.7:4014     , piece_id=65DKIVFKLQVVJEHZOT2P4KYDBHFD6URVGOFPFPABEGTOORENHXBA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.752240113 (769h10m16.623323148s) [noise=true , address=145.79.227.21:4501   , piece_id=KQIHUVS3N56R5HFH2S2I6OE7PBZUXAHRGFMPK57KX7UA5I7BLCSQ, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.752279798 (769h10m16.623362843s) [noise=true , address=77.90.31.12:4616     , piece_id=JCWWA7PGIW23J4OKRJYW33S37FGDTIHBHTCLJSXXA4RH4SE7SWUA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.752291660 (769h10m16.623374695s) [seg_pos={0 0}, upload=672957]: Segment reached optimal threshold of 14 pieces.
Jan  6 16:39:16.752502462 (769h10m16.623585507s) [noise=true , address=145.79.227.6:4279    , piece_id=FNXBVE7QJDR67LLYLBXOCK3OQ7E6O5BIKKXPQMYOSMKIZSNK2EDQ, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uiA6tmZtmHDNaNtkhRtH31q4MJxn3C4bfrLAXmYz5mKmofHoG): protocol: write tcp 188.166.225.170:35010->145.79.227.6:4279: use of closed network connection; write tcp 188.166.225.170:35010->145.79.227.6:4279: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35010->145.79.227.6:4279: use of closed network connection
Jan  6 16:39:16.752510658 (769h10m16.623593693s) [noise=true , address=145.79.227.6:4279    , piece_id=FNXBVE7QJDR67LLYLBXOCK3OQ7E6O5BIKKXPQMYOSMKIZSNK2EDQ, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752633512 (769h10m16.623716557s) [noise=true , address=145.79.227.20:4454   , piece_id=W7E7EUDKE6BTN5I7TSTAMHETYKCXBU7RLYALUHELLOSTOAUFVB2Q, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SafkJPqKHLwMaqyZdLq3yJ1xdjqDtWHFrUwynmegUVwEC7y5J): protocol: expected piece hash; context canceled; EOF
Jan  6 16:39:16.752641587 (769h10m16.623724632s) [noise=true , address=145.79.227.20:4454   , piece_id=W7E7EUDKE6BTN5I7TSTAMHETYKCXBU7RLYALUHELLOSTOAUFVB2Q, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752764281 (769h10m16.623847316s) [noise=true , address=145.79.227.23:4557   , piece_id=QHDDTZ4DZAJCIU3I5HVA54JQLOHCCN73GSJAWZGC5JL6X4WV7ETA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sF7TCu1pL8mrtjoPULJ2VnZQHcJSMmvou2w7YgF58ZHQcnfJT): protocol: write tcp 188.166.225.170:43038->145.79.227.23:4557: use of closed network connection; write tcp 188.166.225.170:43038->145.79.227.23:4557: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43038->145.79.227.23:4557: use of closed network connection
Jan  6 16:39:16.752774249 (769h10m16.623857295s) [noise=true , address=145.79.227.23:4557   , piece_id=QHDDTZ4DZAJCIU3I5HVA54JQLOHCCN73GSJAWZGC5JL6X4WV7ETA, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752797775 (769h10m16.62388081s) [noise=true , address=45.91.116.11:4139    , piece_id=NRCGKCNT7VWR4UUJE7HLXWT23K3BRUV2AHMDPOAVAXQF4UZKIPUA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Tz3HEHACmHU9Gp2W2vHVuF5PXNmdsa5N8nYhwt52HCTjggqRD): protocol: read tcp 188.166.225.170:44870->45.91.116.11:4139: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan  6 16:39:16.752805439 (769h10m16.623888484s) [noise=true , address=45.91.116.11:4139    , piece_id=NRCGKCNT7VWR4UUJE7HLXWT23K3BRUV2AHMDPOAVAXQF4UZKIPUA, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752825017 (769h10m16.623908052s) [seg_pos={0 0}, upload=672957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jan  6 16:39:16.752902604 (769h10m16.623985659s) [upload=672957]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:39:16.752918193 (769h10m16.624001229s) [upload=672957]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:39:17.044336649 (769h10m16.915419684s) [upload=672957]: Done uploading segments...
Jan  6 16:39:17.044357138 (769h10m16.915440173s) [upload=672957]: Upload finished. err: <nil>
Duration: 15.556113538s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
22 out of 60 is failed (threshold=3) [X.XXXX..X.....XXX........X.......X..XX.....XX.X...X.XXX.XX..]
duration:
5.836205728s
trace:
https://jaeger.dev.storj.io/trace/785cfd4b6ea181ea
FAIL (2026-01-06 16:43:50 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, Piece ID: 4D6O4Z7U4RVGP2U5CLZJIGEVELOFQMOLDQWGM7WDNMQQTQ2DDOGQ) context deadline exceeded; piecestore: (Node ID: 1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz, Piece ID: 66K4PIL2RGOLIBPI46C5UMNI33VWUZU45G7UDC5H3COXOTCQ5JTQ) context deadline exceeded
======LOGS======
Jan  6 16:42:34.560046394 (769h13m34.431129439s) [upload=673006]: Starting upload
Jan  6 16:42:34.560148298 (769h13m34.431231333s) [upload=673006]: Done starting upload
Jan  6 16:42:34.560260561 (769h13m34.431343596s) [upload=673006]: Uploading segments...
Jan  6 16:42:34.560413272 (769h13m34.431496317s) [upload=673006]: Got next segment. Inline: false
Jan  6 16:42:34.560449861 (769h13m34.431532896s) [upload=673006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan  6 16:42:34.560454139 (769h13m34.431537175s) [upload=673006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan  6 16:42:34.809245025 (769h13m34.68032806s) [seg_pos={0 0}, upload=673006]: Begin upload segment...
Jan  6 16:42:34.809293067 (769h13m34.680376102s) [seg_pos={0 0}, upload=673006]: Done begin upload segment.
Jan  6 16:42:34.809325459 (769h13m34.680408504s) [noise=true , address=145.79.227.43:4157   , piece_id=66K4PIL2RGOLIBPI46C5UMNI33VWUZU45G7UDC5H3COXOTCQ5JTQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.809333584 (769h13m34.680416629s) [noise=true , address=145.79.227.6:4257    , piece_id=4JDDRJFUBYQHJ65FL5AGQ6U3Q56ECZZOK2YNZ7NXJ3BC4M2XMLBA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.809909852 (769h13m34.680992887s) [noise=true , address=145.79.227.52:4445   , piece_id=B43X42OXZNE2ORQAQRVQPO7CWEMNDBF52RNT7NTC63FSFXHWAJPA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.810218409 (769h13m34.681301444s) [noise=true , address=145.79.227.38:4977   , piece_id=MVW77QHFSSBRLBUVBPX6PVCN2XW5T3CYDLCKGXHTMVNHG74AXBQQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.810258215 (769h13m34.681341251s) [noise=true , address=145.79.227.50:4373   , piece_id=CVYLB5VD3TOIU2DGNVWZC2VLWN6GEYRD24PXEHYHSLOLNMCLI2LQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.810478044 (769h13m34.681561079s) [noise=true , address=45.91.116.12:4225    , piece_id=D7BYWTDRAHH4DQXR6Z7FUBG3ELEG4XLNME4HAJE7E2TCLQNBSDAQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.810623571 (769h13m34.681706606s) [noise=true , address=145.79.227.34:4905   , piece_id=Q6JGJSONJOPM7RCNJ2NFWNGN5YUXRBDKWNG4AXMSKFWHLOIPFRNQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.810717911 (769h13m34.681800946s) [noise=true , address=145.79.227.7:4294    , piece_id=25GOFF4ZH7GZ2A47U2E5GHUUVTKDRXCDHLCQS3PPH3A3SBVAHOQQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811065472 (769h13m34.682148508s) [noise=true , address=145.79.227.16:4909   , piece_id=BEOGAJMSELIPY6VD2IEHU2RFECLSEOHUJL3FYFVTWGCVQFRBSZGA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811168328 (769h13m34.682251363s) [noise=true , address=77.90.31.14:4662     , piece_id=4D6O4Z7U4RVGP2U5CLZJIGEVELOFQMOLDQWGM7WDNMQQTQ2DDOGQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811346107 (769h13m34.682429142s) [noise=true , address=145.79.227.33:4871   , piece_id=DLZGK6BT7YDRARRBQ3KJX7E6ZWFBCXHYYAMVXQIDRJJLK534LW2A, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811599239 (769h13m34.682682284s) [noise=true , address=77.90.31.11:4556     , piece_id=Z5GXRFOQT3O32SM26IHEUE2GMMZLJWQHWK6ZZJCZPRDJTETSKUJQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811628464 (769h13m34.682711509s) [noise=true , address=145.79.227.15:4876   , piece_id=WRB6GEZDONGABWKOBA63IRHHUI7SPQLGIJQ5P65B4WOUKMMZREVA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811932684 (769h13m34.683015729s) [noise=true , address=45.91.116.10:4082    , piece_id=KVQD33U7ZEBUYR5G4QUDUPBRREYLNUFM57KLQNC72WPOVPTPTTAA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.812155878 (769h13m34.683238914s) [noise=true , address=145.79.227.13:4804   , piece_id=MEVDYXECUAP4Z6OACZLXMSH6SWQJC7JNRKEONTQNABB6LQTUQ5DQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.812512708 (769h13m34.683595743s) [noise=true , address=145.79.227.10:4403   , piece_id=TQH4IIC2RVXXHG3D654E5LGO5H4ST3DTZTDCWSH4TOTIQIXYL2RA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:36.475752420 (769h13m36.346835465s) [upload=673006]: Next returned nil segment
Jan  6 16:42:36.475765895 (769h13m36.34684893s) [upload=673006]: Waiting for error group managing segments...
Jan  6 16:42:36.795856688 (769h13m36.666939733s) [noise=true , address=145.79.227.52:4445   , piece_id=B43X42OXZNE2ORQAQRVQPO7CWEMNDBF52RNT7NTC63FSFXHWAJPA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.806051781 (769h13m36.677134816s) [noise=true , address=145.79.227.50:4373   , piece_id=CVYLB5VD3TOIU2DGNVWZC2VLWN6GEYRD24PXEHYHSLOLNMCLI2LQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.815847544 (769h13m36.686930579s) [noise=true , address=145.79.227.7:4294    , piece_id=25GOFF4ZH7GZ2A47U2E5GHUUVTKDRXCDHLCQS3PPH3A3SBVAHOQQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.816611258 (769h13m36.687694293s) [noise=true , address=145.79.227.10:4403   , piece_id=TQH4IIC2RVXXHG3D654E5LGO5H4ST3DTZTDCWSH4TOTIQIXYL2RA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.834582168 (769h13m36.705665203s) [noise=true , address=145.79.227.16:4909   , piece_id=BEOGAJMSELIPY6VD2IEHU2RFECLSEOHUJL3FYFVTWGCVQFRBSZGA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.876254909 (769h13m36.747337944s) [noise=true , address=145.79.227.15:4876   , piece_id=WRB6GEZDONGABWKOBA63IRHHUI7SPQLGIJQ5P65B4WOUKMMZREVA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.881331255 (769h13m36.75241429s) [noise=true , address=145.79.227.6:4257    , piece_id=4JDDRJFUBYQHJ65FL5AGQ6U3Q56ECZZOK2YNZ7NXJ3BC4M2XMLBA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.889252358 (769h13m36.760335393s) [noise=true , address=145.79.227.38:4977   , piece_id=MVW77QHFSSBRLBUVBPX6PVCN2XW5T3CYDLCKGXHTMVNHG74AXBQQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.922061541 (769h13m36.793144576s) [noise=true , address=145.79.227.13:4804   , piece_id=MEVDYXECUAP4Z6OACZLXMSH6SWQJC7JNRKEONTQNABB6LQTUQ5DQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:37.274726429 (769h13m37.145809464s) [noise=true , address=45.91.116.10:4082    , piece_id=KVQD33U7ZEBUYR5G4QUDUPBRREYLNUFM57KLQNC72WPOVPTPTTAA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:37.336706018 (769h13m37.207789063s) [noise=true , address=45.91.116.12:4225    , piece_id=D7BYWTDRAHH4DQXR6Z7FUBG3ELEG4XLNME4HAJE7E2TCLQNBSDAQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:37.926802052 (769h13m37.797885097s) [noise=true , address=77.90.31.14:4662     , piece_id=4D6O4Z7U4RVGP2U5CLZJIGEVELOFQMOLDQWGM7WDNMQQTQ2DDOGQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:38.123587601 (769h13m37.994670656s) [noise=true , address=145.79.227.43:4157   , piece_id=66K4PIL2RGOLIBPI46C5UMNI33VWUZU45G7UDC5H3COXOTCQ5JTQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:38.123609883 (769h13m37.994692918s) [seg_pos={0 0}, upload=673006]: Segment reached optimal threshold of 13 pieces.
Jan  6 16:42:38.123834482 (769h13m37.994917517s) [noise=true , address=145.79.227.33:4871   , piece_id=DLZGK6BT7YDRARRBQ3KJX7E6ZWFBCXHYYAMVXQIDRJJLK534LW2A, seg_pos={0 0}, upload=673006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF
Jan  6 16:42:38.123843408 (769h13m37.994926443s) [noise=true , address=145.79.227.33:4871   , piece_id=DLZGK6BT7YDRARRBQ3KJX7E6ZWFBCXHYYAMVXQIDRJJLK534LW2A, seg_pos={0 0}, upload=673006]: Long tail cancellation due to optimal threshold
Jan  6 16:42:38.123991841 (769h13m37.995074886s) [noise=true , address=145.79.227.34:4905   , piece_id=Q6JGJSONJOPM7RCNJ2NFWNGN5YUXRBDKWNG4AXMSKFWHLOIPFRNQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9): protocol: expected piece hash; context canceled; EOF
Jan  6 16:42:38.124002190 (769h13m37.995085235s) [noise=true , address=145.79.227.34:4905   , piece_id=Q6JGJSONJOPM7RCNJ2NFWNGN5YUXRBDKWNG4AXMSKFWHLOIPFRNQ, seg_pos={0 0}, upload=673006]: Long tail cancellation due to optimal threshold
Jan  6 16:42:38.124039221 (769h13m37.995122266s) [noise=true , address=77.90.31.11:4556     , piece_id=Z5GXRFOQT3O32SM26IHEUE2GMMZLJWQHWK6ZZJCZPRDJTETSKUJQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks): protocol: expected piece hash; context canceled; EOF
Jan  6 16:42:38.124047066 (769h13m37.995130101s) [noise=true , address=77.90.31.11:4556     , piece_id=Z5GXRFOQT3O32SM26IHEUE2GMMZLJWQHWK6ZZJCZPRDJTETSKUJQ, seg_pos={0 0}, upload=673006]: Long tail cancellation due to optimal threshold
Jan  6 16:42:38.124070059 (769h13m37.995153104s) [seg_pos={0 0}, upload=673006]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan  6 16:42:38.124162536 (769h13m37.995245571s) [upload=673006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:42:38.124168918 (769h13m37.995251963s) [upload=673006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:42:38.429966894 (769h13m38.301049939s) [upload=673006]: Done uploading segments...
Jan  6 16:42:38.429991410 (769h13m38.301074445s) [upload=673006]: Upload finished. err: <nil>
Duration: 5.836235724s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
57 out of 60 is failed (threshold=6) [XXXXXXXXXX.XX.XXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
5.001455137s
trace:
https://jaeger.dev.storj.io/trace/29fcb12e5e1c9681
FAIL (2026-01-06 16:43:50 UTC)
orbiter warning: context deadline exceeded
Duration: 5.001476538s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
22 out of 60 is failed (threshold=3) [.X.XXXX..X.....XXX........X.......X..XX.....XX.X...X.XXX.XX.]
duration:
16.685341429s
trace:
https://jaeger.dev.storj.io/trace/785cfd4b6ea181ea
FAIL (2026-01-06 16:43:42 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, Piece ID: 4D6O4Z7U4RVGP2U5CLZJIGEVELOFQMOLDQWGM7WDNMQQTQ2DDOGQ) context deadline exceeded; piecestore: (Node ID: 1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz, Piece ID: 66K4PIL2RGOLIBPI46C5UMNI33VWUZU45G7UDC5H3COXOTCQ5JTQ) context deadline exceeded
======LOGS======
Jan  6 16:42:34.560046394 (769h13m34.431129439s) [upload=673006]: Starting upload
Jan  6 16:42:34.560148298 (769h13m34.431231333s) [upload=673006]: Done starting upload
Jan  6 16:42:34.560260561 (769h13m34.431343596s) [upload=673006]: Uploading segments...
Jan  6 16:42:34.560413272 (769h13m34.431496317s) [upload=673006]: Got next segment. Inline: false
Jan  6 16:42:34.560449861 (769h13m34.431532896s) [upload=673006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan  6 16:42:34.560454139 (769h13m34.431537175s) [upload=673006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan  6 16:42:34.809245025 (769h13m34.68032806s) [seg_pos={0 0}, upload=673006]: Begin upload segment...
Jan  6 16:42:34.809293067 (769h13m34.680376102s) [seg_pos={0 0}, upload=673006]: Done begin upload segment.
Jan  6 16:42:34.809325459 (769h13m34.680408504s) [noise=true , address=145.79.227.43:4157   , piece_id=66K4PIL2RGOLIBPI46C5UMNI33VWUZU45G7UDC5H3COXOTCQ5JTQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.809333584 (769h13m34.680416629s) [noise=true , address=145.79.227.6:4257    , piece_id=4JDDRJFUBYQHJ65FL5AGQ6U3Q56ECZZOK2YNZ7NXJ3BC4M2XMLBA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.809909852 (769h13m34.680992887s) [noise=true , address=145.79.227.52:4445   , piece_id=B43X42OXZNE2ORQAQRVQPO7CWEMNDBF52RNT7NTC63FSFXHWAJPA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.810218409 (769h13m34.681301444s) [noise=true , address=145.79.227.38:4977   , piece_id=MVW77QHFSSBRLBUVBPX6PVCN2XW5T3CYDLCKGXHTMVNHG74AXBQQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.810258215 (769h13m34.681341251s) [noise=true , address=145.79.227.50:4373   , piece_id=CVYLB5VD3TOIU2DGNVWZC2VLWN6GEYRD24PXEHYHSLOLNMCLI2LQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.810478044 (769h13m34.681561079s) [noise=true , address=45.91.116.12:4225    , piece_id=D7BYWTDRAHH4DQXR6Z7FUBG3ELEG4XLNME4HAJE7E2TCLQNBSDAQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.810623571 (769h13m34.681706606s) [noise=true , address=145.79.227.34:4905   , piece_id=Q6JGJSONJOPM7RCNJ2NFWNGN5YUXRBDKWNG4AXMSKFWHLOIPFRNQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.810717911 (769h13m34.681800946s) [noise=true , address=145.79.227.7:4294    , piece_id=25GOFF4ZH7GZ2A47U2E5GHUUVTKDRXCDHLCQS3PPH3A3SBVAHOQQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811065472 (769h13m34.682148508s) [noise=true , address=145.79.227.16:4909   , piece_id=BEOGAJMSELIPY6VD2IEHU2RFECLSEOHUJL3FYFVTWGCVQFRBSZGA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811168328 (769h13m34.682251363s) [noise=true , address=77.90.31.14:4662     , piece_id=4D6O4Z7U4RVGP2U5CLZJIGEVELOFQMOLDQWGM7WDNMQQTQ2DDOGQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811346107 (769h13m34.682429142s) [noise=true , address=145.79.227.33:4871   , piece_id=DLZGK6BT7YDRARRBQ3KJX7E6ZWFBCXHYYAMVXQIDRJJLK534LW2A, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811599239 (769h13m34.682682284s) [noise=true , address=77.90.31.11:4556     , piece_id=Z5GXRFOQT3O32SM26IHEUE2GMMZLJWQHWK6ZZJCZPRDJTETSKUJQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811628464 (769h13m34.682711509s) [noise=true , address=145.79.227.15:4876   , piece_id=WRB6GEZDONGABWKOBA63IRHHUI7SPQLGIJQ5P65B4WOUKMMZREVA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811932684 (769h13m34.683015729s) [noise=true , address=45.91.116.10:4082    , piece_id=KVQD33U7ZEBUYR5G4QUDUPBRREYLNUFM57KLQNC72WPOVPTPTTAA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.812155878 (769h13m34.683238914s) [noise=true , address=145.79.227.13:4804   , piece_id=MEVDYXECUAP4Z6OACZLXMSH6SWQJC7JNRKEONTQNABB6LQTUQ5DQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.812512708 (769h13m34.683595743s) [noise=true , address=145.79.227.10:4403   , piece_id=TQH4IIC2RVXXHG3D654E5LGO5H4ST3DTZTDCWSH4TOTIQIXYL2RA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:36.475752420 (769h13m36.346835465s) [upload=673006]: Next returned nil segment
Jan  6 16:42:36.475765895 (769h13m36.34684893s) [upload=673006]: Waiting for error group managing segments...
Jan  6 16:42:36.795856688 (769h13m36.666939733s) [noise=true , address=145.79.227.52:4445   , piece_id=B43X42OXZNE2ORQAQRVQPO7CWEMNDBF52RNT7NTC63FSFXHWAJPA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.806051781 (769h13m36.677134816s) [noise=true , address=145.79.227.50:4373   , piece_id=CVYLB5VD3TOIU2DGNVWZC2VLWN6GEYRD24PXEHYHSLOLNMCLI2LQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.815847544 (769h13m36.686930579s) [noise=true , address=145.79.227.7:4294    , piece_id=25GOFF4ZH7GZ2A47U2E5GHUUVTKDRXCDHLCQS3PPH3A3SBVAHOQQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.816611258 (769h13m36.687694293s) [noise=true , address=145.79.227.10:4403   , piece_id=TQH4IIC2RVXXHG3D654E5LGO5H4ST3DTZTDCWSH4TOTIQIXYL2RA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.834582168 (769h13m36.705665203s) [noise=true , address=145.79.227.16:4909   , piece_id=BEOGAJMSELIPY6VD2IEHU2RFECLSEOHUJL3FYFVTWGCVQFRBSZGA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.876254909 (769h13m36.747337944s) [noise=true , address=145.79.227.15:4876   , piece_id=WRB6GEZDONGABWKOBA63IRHHUI7SPQLGIJQ5P65B4WOUKMMZREVA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.881331255 (769h13m36.75241429s) [noise=true , address=145.79.227.6:4257    , piece_id=4JDDRJFUBYQHJ65FL5AGQ6U3Q56ECZZOK2YNZ7NXJ3BC4M2XMLBA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.889252358 (769h13m36.760335393s) [noise=true , address=145.79.227.38:4977   , piece_id=MVW77QHFSSBRLBUVBPX6PVCN2XW5T3CYDLCKGXHTMVNHG74AXBQQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.922061541 (769h13m36.793144576s) [noise=true , address=145.79.227.13:4804   , piece_id=MEVDYXECUAP4Z6OACZLXMSH6SWQJC7JNRKEONTQNABB6LQTUQ5DQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:37.274726429 (769h13m37.145809464s) [noise=true , address=45.91.116.10:4082    , piece_id=KVQD33U7ZEBUYR5G4QUDUPBRREYLNUFM57KLQNC72WPOVPTPTTAA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:37.336706018 (769h13m37.207789063s) [noise=true , address=45.91.116.12:4225    , piece_id=D7BYWTDRAHH4DQXR6Z7FUBG3ELEG4XLNME4HAJE7E2TCLQNBSDAQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:37.926802052 (769h13m37.797885097s) [noise=true , address=77.90.31.14:4662     , piece_id=4D6O4Z7U4RVGP2U5CLZJIGEVELOFQMOLDQWGM7WDNMQQTQ2DDOGQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:38.123587601 (769h13m37.994670656s) [noise=true , address=145.79.227.43:4157   , piece_id=66K4PIL2RGOLIBPI46C5UMNI33VWUZU45G7UDC5H3COXOTCQ5JTQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:38.123609883 (769h13m37.994692918s) [seg_pos={0 0}, upload=673006]: Segment reached optimal threshold of 13 pieces.
Jan  6 16:42:38.123834482 (769h13m37.994917517s) [noise=true , address=145.79.227.33:4871   , piece_id=DLZGK6BT7YDRARRBQ3KJX7E6ZWFBCXHYYAMVXQIDRJJLK534LW2A, seg_pos={0 0}, upload=673006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF
Jan  6 16:42:38.123843408 (769h13m37.994926443s) [noise=true , address=145.79.227.33:4871   , piece_id=DLZGK6BT7YDRARRBQ3KJX7E6ZWFBCXHYYAMVXQIDRJJLK534LW2A, seg_pos={0 0}, upload=673006]: Long tail cancellation due to optimal threshold
Jan  6 16:42:38.123991841 (769h13m37.995074886s) [noise=true , address=145.79.227.34:4905   , piece_id=Q6JGJSONJOPM7RCNJ2NFWNGN5YUXRBDKWNG4AXMSKFWHLOIPFRNQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9): protocol: expected piece hash; context canceled; EOF
Jan  6 16:42:38.124002190 (769h13m37.995085235s) [noise=true , address=145.79.227.34:4905   , piece_id=Q6JGJSONJOPM7RCNJ2NFWNGN5YUXRBDKWNG4AXMSKFWHLOIPFRNQ, seg_pos={0 0}, upload=673006]: Long tail cancellation due to optimal threshold
Jan  6 16:42:38.124039221 (769h13m37.995122266s) [noise=true , address=77.90.31.11:4556     , piece_id=Z5GXRFOQT3O32SM26IHEUE2GMMZLJWQHWK6ZZJCZPRDJTETSKUJQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks): protocol: expected piece hash; context canceled; EOF
Jan  6 16:42:38.124047066 (769h13m37.995130101s) [noise=true , address=77.90.31.11:4556     , piece_id=Z5GXRFOQT3O32SM26IHEUE2GMMZLJWQHWK6ZZJCZPRDJTETSKUJQ, seg_pos={0 0}, upload=673006]: Long tail cancellation due to optimal threshold
Jan  6 16:42:38.124070059 (769h13m37.995153104s) [seg_pos={0 0}, upload=673006]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan  6 16:42:38.124162536 (769h13m37.995245571s) [upload=673006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:42:38.124168918 (769h13m37.995251963s) [upload=673006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:42:38.429966894 (769h13m38.301049939s) [upload=673006]: Done uploading segments...
Jan  6 16:42:38.429991410 (769h13m38.301074445s) [upload=673006]: Upload finished. err: <nil>
Duration: 16.685366918s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
2 out of 60 is failed (threshold=3) [......................X......X..............................]
duration:
17.839813099s
trace:
https://jaeger.dev.storj.io/trace/d5e2d8b8127d900
FAIL (2026-01-06 16:43:42 UTC)
orbiter warning: context deadline exceeded
Duration: 17.839837657s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)
antiblip:
2 out of 60 is failed (threshold=3) [.................................................XX.........]
duration:
14.87574855s
trace:
https://jaeger.dev.storj.io/trace/4b3b841c440bfc62
FAIL (2026-01-06 16:43:24 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
JXEZNOOX75ENDV6EBXU4NR4HW4GTZMVA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913312874 (769h5m40.784395909s) [noise=true , address=81.171.14.70:4035    , piece_id=BRXIJD7JX7L37MWMTVPL5KSESQUQEZNRXLBLL26XDMYIJSCRUD5A, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913698277 (769h5m40.784781322s) [noise=true , address=81.171.14.70:4030    , piece_id=WV5BULPUQSAE7DUO2FWY74KF3MIHSQXYC673GNJ7XUBQXXM6KHEQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913862180 (769h5m40.784945215s) [noise=true , address=89.149.201.147:4004  , piece_id=NMPCYNSE3YVIAVDCWRO4VP6IN4DKZ2P4YIFYI6ZDTEU75QSFC4NQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.914346662 (769h5m40.785429707s) [noise=true , address=185.167.204.205:4035 , piece_id=HPHYG44QNKJYFSXEBH6FPYLFXLO6OUIVGDYL5GXJMP3PDNFUZEQA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.914842456 (769h5m40.785925491s) [noise=true , address=81.171.14.70:4034    , piece_id=5LGDXPCHMOQPZGJAEH3ZSPF64RSTVAJRQFZBXVWP5D3Y36YFLUUQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.915307461 (769h5m40.786390506s) [noise=true , address=89.149.201.147:4019  , piece_id=LS3NXHELDI7H6VCTHO6LAGRDDG46ST4IF2XOFZDZ6EK756K5ACHQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.915793817 (769h5m40.786876863s) [noise=true , address=185.167.204.206:4067 , piece_id=GLITDKTEKYV4CFGL3TJXWJDFOBYPFX2UTNJFPUCZDJM3HOWVFS4Q, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.916279973 (769h5m40.787363018s) [noise=true , address=89.149.201.147:4014  , piece_id=KNV6OXYOC2H7LVZCW4ZWQRYRN5ZCMII3OO3FAS3YYQ22574KUCRQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.916759336 (769h5m40.787842371s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.920089677 (769h5m40.791172722s) [noise=true , address=185.167.204.205:4014 , piece_id=XUCDQV4VW4ZZFTSZV2YEDAO5GQFP2ASW6BJFFC735UWEVIR3RNXA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:41.741185400 (769h5m41.612268445s) [upload=672894]: Next returned nil segment
Jan  6 16:34:41.741201020 (769h5m41.612284055s) [upload=672894]: Waiting for error group managing segments...
Jan  6 16:34:42.136024704 (769h5m42.00710774s) [noise=true , address=185.167.204.205:4036 , piece_id=4XKAIZPEJZAVAS7P2QILKY7XIDIIRIRWH7HTN5OVXUHAL2Y3H4SQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.137413950 (769h5m42.008496995s) [noise=true , address=185.167.204.206:4051 , piece_id=XWKJM6ZIHISLSESJ2ZQQQMZTGMK5MHH6L4ZD5RXJ3SMAVASVVVOA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.137527506 (769h5m42.008610541s) [noise=true , address=185.167.204.206:4049 , piece_id=4T665GS7XTBVEHUVT6SFIQW4F3TTBPDKZ57PKQHHHWWKC4P4JQPQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.138183485 (769h5m42.00926653s) [noise=true , address=185.167.204.206:4048 , piece_id=ZGQYUYRK5PK24UF4PYT67FHNPR4GVFDCJ2OEGRFENLB2GYR2LFVQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.139902609 (769h5m42.010985644s) [noise=true , address=185.167.204.206:4056 , piece_id=KYHDW5RLJTZCURXCFIP7HN3ZYLPPHK7FTO6IIGQ5VJJUPF4ICQXA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.140277843 (769h5m42.011360878s) [noise=true , address=185.167.204.206:4067 , piece_id=GLITDKTEKYV4CFGL3TJXWJDFOBYPFX2UTNJFPUCZDJM3HOWVFS4Q, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.142394303 (769h5m42.013477348s) [noise=true , address=185.167.204.205:4021 , piece_id=U4X3NZRT7MHG2JTNRVDRYIC72UDMHUQUCY4VFW7JFVLLVACZN7OA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.179529514 (769h5m42.050612549s) [noise=true , address=185.167.204.206:4053 , piece_id=HBFCBJCOLPJLNE4INHP4JXEZNOOX75ENDV6EBXU4NR4HW4GTZMVA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.194387988 (769h5m42.065471023s) [noise=true , address=185.167.204.205:4035 , piece_id=HPHYG44QNKJYFSXEBH6FPYLFXLO6OUIVGDYL5GXJMP3PDNFUZEQA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.197860059 (769h5m42.068943094s) [noise=true , address=185.167.204.206:4064 , piece_id=JGCXE3QUORVKLRIWEXFZUCT4XUF3DRVA35ZKQU4DQODLZBOH5I4A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.827615690 (769h5m42.698698725s) [noise=true , address=81.171.14.70:4041    , piece_id=NLGMNQJDFISSAHBX3EC6WMWJLP74FMR4MRPUIX4GCCCQRG5RRRCQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.917676689 (769h5m42.788759725s) [noise=true , address=81.171.14.70:4034    , piece_id=5LGDXPCHMOQPZGJAEH3ZSPF64RSTVAJRQFZBXVWP5D3Y36YFLUUQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.932213350 (769h5m42.803296385s) [noise=true , address=81.171.14.70:4030    , piece_id=WV5BULPUQSAE7DUO2FWY74KF3MIHSQXYC673GNJ7XUBQXXM6KHEQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.956051121 (769h5m42.827134196s) [noise=true , address=81.171.14.70:4031    , piece_id=OJKJHNVWTPDITZC5CTQNB7UFPIVZDPE54ZXS4LWRM3XHYAB6443A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.005259852 (769h5m42.876342887s) [noise=true , address=185.167.204.205:4019 , piece_id=I7OP7AQS2DC3C7GZWG5RVJSLDKAZD2434WMBPRYMZFVAJYJG3JHQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.052708792 (769h5m42.923791837s) [noise=true , address=81.171.14.70:4046    , piece_id=OV3PMCD6UIFNSQNTCOS3GEQC4676KFJOUH5ZX576OFOYC5EXCZPQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.639749858 (769h5m43.510832903s) [noise=true , address=89.149.201.147:4004  , piece_id=NMPCYNSE3YVIAVDCWRO4VP6IN4DKZ2P4YIFYI6ZDTEU75QSFC4NQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.719270598 (769h5m43.590353643s) [noise=true , address=89.149.201.147:4019  , piece_id=LS3NXHELDI7H6VCTHO6LAGRDDG46ST4IF2XOFZDZ6EK756K5ACHQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.892052420 (769h5m43.763135466s) [noise=true , address=81.171.14.70:4035    , piece_id=BRXIJD7JX7L37MWMTVPL5KSESQUQEZNRXLBLL26XDMYIJSCRUD5A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.901787498 (769h5m43.772870533s) [noise=true , address=81.171.14.70:4039    , piece_id=VFXOSS5DRL43RDJGPLSH3667HWKAF5ZNC2CF5BA7TDR4NSCG75HA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.019887208 (769h5m43.890970243s) [noise=true , address=185.167.204.206:4050 , piece_id=KSCVNGXUGFV7XRNU5MFIKLLQGZ5BNHTIXQLVHWPMITZOQUA23DGQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.510337921 (769h5m44.381420957s) [noise=true , address=185.167.204.205:4028 , piece_id=QPB4EPPEENW277S4SBIOCWWMBLIE6GKCGEM3TV3MXN6SYFMQAI7Q, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.692198242 (769h5m44.563281277s) [noise=true , address=89.149.201.147:4010  , piece_id=C57ORVBIZE4QUWCVG3OVG3KW2Z2GYVWUXEOYG5R53KV2WUQLR7JQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.394574949 (769h5m46.265657984s) [noise=true , address=89.149.201.147:4014  , piece_id=KNV6OXYOC2H7LVZCW4ZWQRYRN5ZCMII3OO3FAS3YYQ22574KUCRQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.543864889 (769h5m46.414947934s) [noise=true , address=185.167.204.205:4014 , piece_id=XUCDQV4VW4ZZFTSZV2YEDAO5GQFP2ASW6BJFFC735UWEVIR3RNXA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.846238946 (769h5m46.717321982s) [noise=true , address=185.167.204.205:4006 , piece_id=W3XXEI7MG37VJHHK2CECNKIIHB7WSQABGVB5W5ZGAWSTFQ4NQMEQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.846258273 (769h5m46.717341308s) [seg_pos={0 0}, upload=672894]: Segment reached optimal threshold of 26 pieces.
Jan  6 16:34:46.846448986 (769h5m46.717532021s) [noise=true , address=185.167.204.206:4072 , piece_id=EJ4XCLNDZETIFIKE3FK2Q4NA35KT5LOTNGE253MKUCAIGH4FZD5A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v): protocol: expected piece hash; context canceled; EOF
Jan  6 16:34:46.846454998 (769h5m46.717538033s) [noise=true , address=185.167.204.206:4072 , piece_id=EJ4XCLNDZETIFIKE3FK2Q4NA35KT5LOTNGE253MKUCAIGH4FZD5A, seg_pos={0 0}, upload=672894]: Long tail cancellation due to optimal threshold
Jan  6 16:34:46.846579175 (769h5m46.71766221s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK): protocol: expected piece hash; context canceled; EOF
Jan  6 16:34:46.846595475 (769h5m46.71767851s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Long tail cancellation due to optimal threshold
Jan  6 16:34:46.846620453 (769h5m46.717703488s) [seg_pos={0 0}, upload=672894]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan  6 16:34:46.846729380 (769h5m46.717812415s) [upload=672894]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:34:46.846734600 (769h5m46.717817635s) [upload=672894]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:34:47.079572108 (769h5m46.950655143s) [upload=672894]: Done uploading segments...
Jan  6 16:34:47.079590854 (769h5m46.950673889s) [upload=672894]: Upload finished. err: <nil>
Duration: 14.875781694s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
11 out of 60 is failed (threshold=3) [......................X.X....X.X.X....X.X...X....XXX........]
duration:
10.691466199s
trace:
https://jaeger.dev.storj.io/trace/73dd0c7413e4b545
FAIL (2026-01-06 16:43:23 UTC)
orbiter warning: context deadline exceeded
Duration: 10.69151914s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
57 out of 60 is failed (threshold=6) [XXXXXXXXXXX.XX.XXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
5.001149019s
trace:
https://jaeger.dev.storj.io/trace/7b1da97514d95cb7
FAIL (2026-01-06 16:43:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001261023s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
13 out of 60 is failed (threshold=3) [X......X..X.X.X........X.....XX......X..........XX...X.X....]
duration:
15.331353821s
trace:
https://jaeger.dev.storj.io/trace/76df23b0035fe317
FAIL (2026-01-06 16:43:04 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
g piece...
Jan  6 16:39:13.319055742 (769h10m13.190138787s) [noise=true , address=77.90.31.12:4616     , piece_id=JCWWA7PGIW23J4OKRJYW33S37FGDTIHBHTCLJSXXA4RH4SE7SWUA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.319538962 (769h10m13.190621997s) [noise=true , address=45.91.116.7:4014     , piece_id=65DKIVFKLQVVJEHZOT2P4KYDBHFD6URVGOFPFPABEGTOORENHXBA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.320135148 (769h10m13.191218193s) [noise=true , address=77.90.31.14:4668     , piece_id=QYCFWBXHW7BYLILL76W5MDFYQJKG6VZIZOYS3BF4OPDZLMJZXKVQ, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.320651351 (769h10m13.191734396s) [noise=true , address=145.79.227.11:4736   , piece_id=CTSU3X6OZJFWJSCLJBJ3EBGALWWGZL62ZJN436LKICHA4MLIQ2ZA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.321172283 (769h10m13.192255328s) [noise=true , address=64.49.13.27:4968     , piece_id=X4RAE7BUCM2R5NDMCCLH4TRGRCCXJBGUXZRE4QSNCFWNCJXLBSVA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.323456752 (769h10m13.194539787s) [noise=true , address=145.79.227.12:4773   , piece_id=JXMIF6EZKPD7EXZ646Z4XM6PG6I5P3EPTBJUWYQX4LMGAUFJT4EA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.325413879 (769h10m13.196496924s) [noise=true , address=145.79.227.33:4884   , piece_id=E5YFUCC75VTQBLZH3T5Q6VWEMRYAVOXNFKZQCPEP3ES64YCTXWPA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.327557260 (769h10m13.198640305s) [noise=true , address=145.79.227.6:4279    , piece_id=FNXBVE7QJDR67LLYLBXOCK3OQ7E6O5BIKKXPQMYOSMKIZSNK2EDQ, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.329566295 (769h10m13.20064934s) [noise=true , address=45.91.116.9:4282     , piece_id=GC72TML4SY7YTFQW4G3CVTOMGW5HWQLCWHBFQ3TKSZSUJUR252WA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.330791288 (769h10m13.201874323s) [noise=true , address=45.91.116.11:4139    , piece_id=NRCGKCNT7VWR4UUJE7HLXWT23K3BRUV2AHMDPOAVAXQF4UZKIPUA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.332349425 (769h10m13.20343246s) [noise=true , address=145.79.227.49:4371   , piece_id=YNFO36VNDUUPFQO4OFHIUEWUZWLYM2BUD7NF6VQEVWZHNCDSM6FA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.332771599 (769h10m13.203854634s) [noise=true , address=145.79.227.23:4557   , piece_id=QHDDTZ4DZAJCIU3I5HVA54JQLOHCCN73GSJAWZGC5JL6X4WV7ETA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.333853448 (769h10m13.204936494s) [noise=true , address=145.79.227.14:4843   , piece_id=GSANAVILLP6UTITEPDCBK5VHTTK7D2SBT7FQQI77H35I3SLNRVKA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.336038750 (769h10m13.207121785s) [noise=true , address=45.91.116.10:4107    , piece_id=I5OP5FXQBURDQG2LWEKGVZUBX7VV3CLD6Z2ZKKDK5UCX2RGFX3BA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:14.793189300 (769h10m14.664272335s) [upload=672957]: Next returned nil segment
Jan  6 16:39:14.793201272 (769h10m14.664284307s) [upload=672957]: Waiting for error group managing segments...
Jan  6 16:39:15.209058789 (769h10m15.080141834s) [noise=true , address=145.79.227.13:4835   , piece_id=MMTT3F5VEI2NUZ4T2QWJGNCPIB3O4EGLYANWMCFK6MC3YLQX3TXA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.251874898 (769h10m15.122957933s) [noise=true , address=145.79.227.14:4843   , piece_id=GSANAVILLP6UTITEPDCBK5VHTTK7D2SBT7FQQI77H35I3SLNRVKA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.299718689 (769h10m15.170801724s) [noise=true , address=145.79.227.11:4736   , piece_id=CTSU3X6OZJFWJSCLJBJ3EBGALWWGZL62ZJN436LKICHA4MLIQ2ZA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.351276473 (769h10m15.222359518s) [noise=true , address=145.79.227.12:4773   , piece_id=JXMIF6EZKPD7EXZ646Z4XM6PG6I5P3EPTBJUWYQX4LMGAUFJT4EA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.423449492 (769h10m15.294532527s) [noise=true , address=64.49.13.27:4968     , piece_id=X4RAE7BUCM2R5NDMCCLH4TRGRCCXJBGUXZRE4QSNCFWNCJXLBSVA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.580777038 (769h10m15.451860083s) [noise=true , address=77.90.31.14:4668     , piece_id=QYCFWBXHW7BYLILL76W5MDFYQJKG6VZIZOYS3BF4OPDZLMJZXKVQ, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.798468394 (769h10m15.66955143s) [noise=true , address=45.91.116.10:4107    , piece_id=I5OP5FXQBURDQG2LWEKGVZUBX7VV3CLD6Z2ZKKDK5UCX2RGFX3BA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.083877398 (769h10m15.954960443s) [noise=true , address=45.91.116.9:4282     , piece_id=GC72TML4SY7YTFQW4G3CVTOMGW5HWQLCWHBFQ3TKSZSUJUR252WA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.143050622 (769h10m16.014133687s) [noise=true , address=145.79.227.32:4865   , piece_id=Z5ZJEDUJG376CVJM4EDK6JPRJK2U5OR6RDD4YG5X4MXH4WKSZYSA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.194307403 (769h10m16.065390438s) [noise=true , address=145.79.227.33:4884   , piece_id=E5YFUCC75VTQBLZH3T5Q6VWEMRYAVOXNFKZQCPEP3ES64YCTXWPA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.244019653 (769h10m16.115102688s) [noise=true , address=145.79.227.49:4371   , piece_id=YNFO36VNDUUPFQO4OFHIUEWUZWLYM2BUD7NF6VQEVWZHNCDSM6FA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.587195683 (769h10m16.458278718s) [noise=true , address=45.91.116.7:4014     , piece_id=65DKIVFKLQVVJEHZOT2P4KYDBHFD6URVGOFPFPABEGTOORENHXBA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.752240113 (769h10m16.623323148s) [noise=true , address=145.79.227.21:4501   , piece_id=KQIHUVS3N56R5HFH2S2I6OE7PBZUXAHRGFMPK57KX7UA5I7BLCSQ, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.752279798 (769h10m16.623362843s) [noise=true , address=77.90.31.12:4616     , piece_id=JCWWA7PGIW23J4OKRJYW33S37FGDTIHBHTCLJSXXA4RH4SE7SWUA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.752291660 (769h10m16.623374695s) [seg_pos={0 0}, upload=672957]: Segment reached optimal threshold of 14 pieces.
Jan  6 16:39:16.752502462 (769h10m16.623585507s) [noise=true , address=145.79.227.6:4279    , piece_id=FNXBVE7QJDR67LLYLBXOCK3OQ7E6O5BIKKXPQMYOSMKIZSNK2EDQ, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uiA6tmZtmHDNaNtkhRtH31q4MJxn3C4bfrLAXmYz5mKmofHoG): protocol: write tcp 188.166.225.170:35010->145.79.227.6:4279: use of closed network connection; write tcp 188.166.225.170:35010->145.79.227.6:4279: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35010->145.79.227.6:4279: use of closed network connection
Jan  6 16:39:16.752510658 (769h10m16.623593693s) [noise=true , address=145.79.227.6:4279    , piece_id=FNXBVE7QJDR67LLYLBXOCK3OQ7E6O5BIKKXPQMYOSMKIZSNK2EDQ, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752633512 (769h10m16.623716557s) [noise=true , address=145.79.227.20:4454   , piece_id=W7E7EUDKE6BTN5I7TSTAMHETYKCXBU7RLYALUHELLOSTOAUFVB2Q, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SafkJPqKHLwMaqyZdLq3yJ1xdjqDtWHFrUwynmegUVwEC7y5J): protocol: expected piece hash; context canceled; EOF
Jan  6 16:39:16.752641587 (769h10m16.623724632s) [noise=true , address=145.79.227.20:4454   , piece_id=W7E7EUDKE6BTN5I7TSTAMHETYKCXBU7RLYALUHELLOSTOAUFVB2Q, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752764281 (769h10m16.623847316s) [noise=true , address=145.79.227.23:4557   , piece_id=QHDDTZ4DZAJCIU3I5HVA54JQLOHCCN73GSJAWZGC5JL6X4WV7ETA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sF7TCu1pL8mrtjoPULJ2VnZQHcJSMmvou2w7YgF58ZHQcnfJT): protocol: write tcp 188.166.225.170:43038->145.79.227.23:4557: use of closed network connection; write tcp 188.166.225.170:43038->145.79.227.23:4557: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43038->145.79.227.23:4557: use of closed network connection
Jan  6 16:39:16.752774249 (769h10m16.623857295s) [noise=true , address=145.79.227.23:4557   , piece_id=QHDDTZ4DZAJCIU3I5HVA54JQLOHCCN73GSJAWZGC5JL6X4WV7ETA, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752797775 (769h10m16.62388081s) [noise=true , address=45.91.116.11:4139    , piece_id=NRCGKCNT7VWR4UUJE7HLXWT23K3BRUV2AHMDPOAVAXQF4UZKIPUA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Tz3HEHACmHU9Gp2W2vHVuF5PXNmdsa5N8nYhwt52HCTjggqRD): protocol: read tcp 188.166.225.170:44870->45.91.116.11:4139: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan  6 16:39:16.752805439 (769h10m16.623888484s) [noise=true , address=45.91.116.11:4139    , piece_id=NRCGKCNT7VWR4UUJE7HLXWT23K3BRUV2AHMDPOAVAXQF4UZKIPUA, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752825017 (769h10m16.623908052s) [seg_pos={0 0}, upload=672957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jan  6 16:39:16.752902604 (769h10m16.623985659s) [upload=672957]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:39:16.752918193 (769h10m16.624001229s) [upload=672957]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:39:17.044336649 (769h10m16.915419684s) [upload=672957]: Done uploading segments...
Jan  6 16:39:17.044357138 (769h10m16.915440173s) [upload=672957]: Upload finished. err: <nil>
Duration: 15.331378667s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
22 out of 60 is failed (threshold=3) [..X.XXXX..X.....XXX........X.......X..XX.....XX.X...X.XXX.XX]
duration:
30.003442171s
trace:
https://jaeger.dev.storj.io/trace/785cfd4b6ea181ea
FAIL (2026-01-06 16:43:03 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, Piece ID: 4D6O4Z7U4RVGP2U5CLZJIGEVELOFQMOLDQWGM7WDNMQQTQ2DDOGQ) context deadline exceeded; piecestore: (Node ID: 1ZrzADk2mKG4R9tjxUKjQeJvb4WmfjLksGFyvstHRfYbBC7XNz, Piece ID: 66K4PIL2RGOLIBPI46C5UMNI33VWUZU45G7UDC5H3COXOTCQ5JTQ) context deadline exceeded
======LOGS======
Jan  6 16:42:34.560046394 (769h13m34.431129439s) [upload=673006]: Starting upload
Jan  6 16:42:34.560148298 (769h13m34.431231333s) [upload=673006]: Done starting upload
Jan  6 16:42:34.560260561 (769h13m34.431343596s) [upload=673006]: Uploading segments...
Jan  6 16:42:34.560413272 (769h13m34.431496317s) [upload=673006]: Got next segment. Inline: false
Jan  6 16:42:34.560449861 (769h13m34.431532896s) [upload=673006]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan  6 16:42:34.560454139 (769h13m34.431537175s) [upload=673006]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan  6 16:42:34.809245025 (769h13m34.68032806s) [seg_pos={0 0}, upload=673006]: Begin upload segment...
Jan  6 16:42:34.809293067 (769h13m34.680376102s) [seg_pos={0 0}, upload=673006]: Done begin upload segment.
Jan  6 16:42:34.809325459 (769h13m34.680408504s) [noise=true , address=145.79.227.43:4157   , piece_id=66K4PIL2RGOLIBPI46C5UMNI33VWUZU45G7UDC5H3COXOTCQ5JTQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.809333584 (769h13m34.680416629s) [noise=true , address=145.79.227.6:4257    , piece_id=4JDDRJFUBYQHJ65FL5AGQ6U3Q56ECZZOK2YNZ7NXJ3BC4M2XMLBA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.809909852 (769h13m34.680992887s) [noise=true , address=145.79.227.52:4445   , piece_id=B43X42OXZNE2ORQAQRVQPO7CWEMNDBF52RNT7NTC63FSFXHWAJPA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.810218409 (769h13m34.681301444s) [noise=true , address=145.79.227.38:4977   , piece_id=MVW77QHFSSBRLBUVBPX6PVCN2XW5T3CYDLCKGXHTMVNHG74AXBQQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.810258215 (769h13m34.681341251s) [noise=true , address=145.79.227.50:4373   , piece_id=CVYLB5VD3TOIU2DGNVWZC2VLWN6GEYRD24PXEHYHSLOLNMCLI2LQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.810478044 (769h13m34.681561079s) [noise=true , address=45.91.116.12:4225    , piece_id=D7BYWTDRAHH4DQXR6Z7FUBG3ELEG4XLNME4HAJE7E2TCLQNBSDAQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.810623571 (769h13m34.681706606s) [noise=true , address=145.79.227.34:4905   , piece_id=Q6JGJSONJOPM7RCNJ2NFWNGN5YUXRBDKWNG4AXMSKFWHLOIPFRNQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.810717911 (769h13m34.681800946s) [noise=true , address=145.79.227.7:4294    , piece_id=25GOFF4ZH7GZ2A47U2E5GHUUVTKDRXCDHLCQS3PPH3A3SBVAHOQQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811065472 (769h13m34.682148508s) [noise=true , address=145.79.227.16:4909   , piece_id=BEOGAJMSELIPY6VD2IEHU2RFECLSEOHUJL3FYFVTWGCVQFRBSZGA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811168328 (769h13m34.682251363s) [noise=true , address=77.90.31.14:4662     , piece_id=4D6O4Z7U4RVGP2U5CLZJIGEVELOFQMOLDQWGM7WDNMQQTQ2DDOGQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811346107 (769h13m34.682429142s) [noise=true , address=145.79.227.33:4871   , piece_id=DLZGK6BT7YDRARRBQ3KJX7E6ZWFBCXHYYAMVXQIDRJJLK534LW2A, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811599239 (769h13m34.682682284s) [noise=true , address=77.90.31.11:4556     , piece_id=Z5GXRFOQT3O32SM26IHEUE2GMMZLJWQHWK6ZZJCZPRDJTETSKUJQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811628464 (769h13m34.682711509s) [noise=true , address=145.79.227.15:4876   , piece_id=WRB6GEZDONGABWKOBA63IRHHUI7SPQLGIJQ5P65B4WOUKMMZREVA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.811932684 (769h13m34.683015729s) [noise=true , address=45.91.116.10:4082    , piece_id=KVQD33U7ZEBUYR5G4QUDUPBRREYLNUFM57KLQNC72WPOVPTPTTAA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.812155878 (769h13m34.683238914s) [noise=true , address=145.79.227.13:4804   , piece_id=MEVDYXECUAP4Z6OACZLXMSH6SWQJC7JNRKEONTQNABB6LQTUQ5DQ, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:34.812512708 (769h13m34.683595743s) [noise=true , address=145.79.227.10:4403   , piece_id=TQH4IIC2RVXXHG3D654E5LGO5H4ST3DTZTDCWSH4TOTIQIXYL2RA, seg_pos={0 0}, upload=673006]: Uploading piece...
Jan  6 16:42:36.475752420 (769h13m36.346835465s) [upload=673006]: Next returned nil segment
Jan  6 16:42:36.475765895 (769h13m36.34684893s) [upload=673006]: Waiting for error group managing segments...
Jan  6 16:42:36.795856688 (769h13m36.666939733s) [noise=true , address=145.79.227.52:4445   , piece_id=B43X42OXZNE2ORQAQRVQPO7CWEMNDBF52RNT7NTC63FSFXHWAJPA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.806051781 (769h13m36.677134816s) [noise=true , address=145.79.227.50:4373   , piece_id=CVYLB5VD3TOIU2DGNVWZC2VLWN6GEYRD24PXEHYHSLOLNMCLI2LQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.815847544 (769h13m36.686930579s) [noise=true , address=145.79.227.7:4294    , piece_id=25GOFF4ZH7GZ2A47U2E5GHUUVTKDRXCDHLCQS3PPH3A3SBVAHOQQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.816611258 (769h13m36.687694293s) [noise=true , address=145.79.227.10:4403   , piece_id=TQH4IIC2RVXXHG3D654E5LGO5H4ST3DTZTDCWSH4TOTIQIXYL2RA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.834582168 (769h13m36.705665203s) [noise=true , address=145.79.227.16:4909   , piece_id=BEOGAJMSELIPY6VD2IEHU2RFECLSEOHUJL3FYFVTWGCVQFRBSZGA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.876254909 (769h13m36.747337944s) [noise=true , address=145.79.227.15:4876   , piece_id=WRB6GEZDONGABWKOBA63IRHHUI7SPQLGIJQ5P65B4WOUKMMZREVA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.881331255 (769h13m36.75241429s) [noise=true , address=145.79.227.6:4257    , piece_id=4JDDRJFUBYQHJ65FL5AGQ6U3Q56ECZZOK2YNZ7NXJ3BC4M2XMLBA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.889252358 (769h13m36.760335393s) [noise=true , address=145.79.227.38:4977   , piece_id=MVW77QHFSSBRLBUVBPX6PVCN2XW5T3CYDLCKGXHTMVNHG74AXBQQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:36.922061541 (769h13m36.793144576s) [noise=true , address=145.79.227.13:4804   , piece_id=MEVDYXECUAP4Z6OACZLXMSH6SWQJC7JNRKEONTQNABB6LQTUQ5DQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:37.274726429 (769h13m37.145809464s) [noise=true , address=45.91.116.10:4082    , piece_id=KVQD33U7ZEBUYR5G4QUDUPBRREYLNUFM57KLQNC72WPOVPTPTTAA, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:37.336706018 (769h13m37.207789063s) [noise=true , address=45.91.116.12:4225    , piece_id=D7BYWTDRAHH4DQXR6Z7FUBG3ELEG4XLNME4HAJE7E2TCLQNBSDAQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:37.926802052 (769h13m37.797885097s) [noise=true , address=77.90.31.14:4662     , piece_id=4D6O4Z7U4RVGP2U5CLZJIGEVELOFQMOLDQWGM7WDNMQQTQ2DDOGQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:38.123587601 (769h13m37.994670656s) [noise=true , address=145.79.227.43:4157   , piece_id=66K4PIL2RGOLIBPI46C5UMNI33VWUZU45G7UDC5H3COXOTCQ5JTQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: <nil>
Jan  6 16:42:38.123609883 (769h13m37.994692918s) [seg_pos={0 0}, upload=673006]: Segment reached optimal threshold of 13 pieces.
Jan  6 16:42:38.123834482 (769h13m37.994917517s) [noise=true , address=145.79.227.33:4871   , piece_id=DLZGK6BT7YDRARRBQ3KJX7E6ZWFBCXHYYAMVXQIDRJJLK534LW2A, seg_pos={0 0}, upload=673006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF
Jan  6 16:42:38.123843408 (769h13m37.994926443s) [noise=true , address=145.79.227.33:4871   , piece_id=DLZGK6BT7YDRARRBQ3KJX7E6ZWFBCXHYYAMVXQIDRJJLK534LW2A, seg_pos={0 0}, upload=673006]: Long tail cancellation due to optimal threshold
Jan  6 16:42:38.123991841 (769h13m37.995074886s) [noise=true , address=145.79.227.34:4905   , piece_id=Q6JGJSONJOPM7RCNJ2NFWNGN5YUXRBDKWNG4AXMSKFWHLOIPFRNQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ffKESNSVpXQzbwJinP6vvgE6zoF3RFz2NRUuuJHvd7bvk1gc9): protocol: expected piece hash; context canceled; EOF
Jan  6 16:42:38.124002190 (769h13m37.995085235s) [noise=true , address=145.79.227.34:4905   , piece_id=Q6JGJSONJOPM7RCNJ2NFWNGN5YUXRBDKWNG4AXMSKFWHLOIPFRNQ, seg_pos={0 0}, upload=673006]: Long tail cancellation due to optimal threshold
Jan  6 16:42:38.124039221 (769h13m37.995122266s) [noise=true , address=77.90.31.11:4556     , piece_id=Z5GXRFOQT3O32SM26IHEUE2GMMZLJWQHWK6ZZJCZPRDJTETSKUJQ, seg_pos={0 0}, upload=673006]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1QNr5B8kdrSa59rHAb82TcvSYpru7m7zbdxXqqxw3FpcFyfpks): protocol: expected piece hash; context canceled; EOF
Jan  6 16:42:38.124047066 (769h13m37.995130101s) [noise=true , address=77.90.31.11:4556     , piece_id=Z5GXRFOQT3O32SM26IHEUE2GMMZLJWQHWK6ZZJCZPRDJTETSKUJQ, seg_pos={0 0}, upload=673006]: Long tail cancellation due to optimal threshold
Jan  6 16:42:38.124070059 (769h13m37.995153104s) [seg_pos={0 0}, upload=673006]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan  6 16:42:38.124162536 (769h13m37.995245571s) [upload=673006]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:42:38.124168918 (769h13m37.995251963s) [upload=673006]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:42:38.429966894 (769h13m38.301049939s) [upload=673006]: Done uploading segments...
Jan  6 16:42:38.429991410 (769h13m38.301074445s) [upload=673006]: Upload finished. err: <nil>
Duration: 30.003459403s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
12 out of 60 is failed (threshold=3) [X......................X.X....X.X.X....X.X...X....XXX.......]
duration:
18.020246523s
trace:
https://jaeger.dev.storj.io/trace/73dd0c7413e4b545
FAIL (2026-01-06 16:42:52 UTC)
orbiter warning: context deadline exceeded
Duration: 18.020270419s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)
antiblip:
2 out of 60 is failed (threshold=3) [..................................................XX........]
duration:
21.828424216s
trace:
https://jaeger.dev.storj.io/trace/4b3b841c440bfc62
FAIL (2026-01-06 16:42:48 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
JXEZNOOX75ENDV6EBXU4NR4HW4GTZMVA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913312874 (769h5m40.784395909s) [noise=true , address=81.171.14.70:4035    , piece_id=BRXIJD7JX7L37MWMTVPL5KSESQUQEZNRXLBLL26XDMYIJSCRUD5A, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913698277 (769h5m40.784781322s) [noise=true , address=81.171.14.70:4030    , piece_id=WV5BULPUQSAE7DUO2FWY74KF3MIHSQXYC673GNJ7XUBQXXM6KHEQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913862180 (769h5m40.784945215s) [noise=true , address=89.149.201.147:4004  , piece_id=NMPCYNSE3YVIAVDCWRO4VP6IN4DKZ2P4YIFYI6ZDTEU75QSFC4NQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.914346662 (769h5m40.785429707s) [noise=true , address=185.167.204.205:4035 , piece_id=HPHYG44QNKJYFSXEBH6FPYLFXLO6OUIVGDYL5GXJMP3PDNFUZEQA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.914842456 (769h5m40.785925491s) [noise=true , address=81.171.14.70:4034    , piece_id=5LGDXPCHMOQPZGJAEH3ZSPF64RSTVAJRQFZBXVWP5D3Y36YFLUUQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.915307461 (769h5m40.786390506s) [noise=true , address=89.149.201.147:4019  , piece_id=LS3NXHELDI7H6VCTHO6LAGRDDG46ST4IF2XOFZDZ6EK756K5ACHQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.915793817 (769h5m40.786876863s) [noise=true , address=185.167.204.206:4067 , piece_id=GLITDKTEKYV4CFGL3TJXWJDFOBYPFX2UTNJFPUCZDJM3HOWVFS4Q, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.916279973 (769h5m40.787363018s) [noise=true , address=89.149.201.147:4014  , piece_id=KNV6OXYOC2H7LVZCW4ZWQRYRN5ZCMII3OO3FAS3YYQ22574KUCRQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.916759336 (769h5m40.787842371s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.920089677 (769h5m40.791172722s) [noise=true , address=185.167.204.205:4014 , piece_id=XUCDQV4VW4ZZFTSZV2YEDAO5GQFP2ASW6BJFFC735UWEVIR3RNXA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:41.741185400 (769h5m41.612268445s) [upload=672894]: Next returned nil segment
Jan  6 16:34:41.741201020 (769h5m41.612284055s) [upload=672894]: Waiting for error group managing segments...
Jan  6 16:34:42.136024704 (769h5m42.00710774s) [noise=true , address=185.167.204.205:4036 , piece_id=4XKAIZPEJZAVAS7P2QILKY7XIDIIRIRWH7HTN5OVXUHAL2Y3H4SQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.137413950 (769h5m42.008496995s) [noise=true , address=185.167.204.206:4051 , piece_id=XWKJM6ZIHISLSESJ2ZQQQMZTGMK5MHH6L4ZD5RXJ3SMAVASVVVOA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.137527506 (769h5m42.008610541s) [noise=true , address=185.167.204.206:4049 , piece_id=4T665GS7XTBVEHUVT6SFIQW4F3TTBPDKZ57PKQHHHWWKC4P4JQPQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.138183485 (769h5m42.00926653s) [noise=true , address=185.167.204.206:4048 , piece_id=ZGQYUYRK5PK24UF4PYT67FHNPR4GVFDCJ2OEGRFENLB2GYR2LFVQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.139902609 (769h5m42.010985644s) [noise=true , address=185.167.204.206:4056 , piece_id=KYHDW5RLJTZCURXCFIP7HN3ZYLPPHK7FTO6IIGQ5VJJUPF4ICQXA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.140277843 (769h5m42.011360878s) [noise=true , address=185.167.204.206:4067 , piece_id=GLITDKTEKYV4CFGL3TJXWJDFOBYPFX2UTNJFPUCZDJM3HOWVFS4Q, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.142394303 (769h5m42.013477348s) [noise=true , address=185.167.204.205:4021 , piece_id=U4X3NZRT7MHG2JTNRVDRYIC72UDMHUQUCY4VFW7JFVLLVACZN7OA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.179529514 (769h5m42.050612549s) [noise=true , address=185.167.204.206:4053 , piece_id=HBFCBJCOLPJLNE4INHP4JXEZNOOX75ENDV6EBXU4NR4HW4GTZMVA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.194387988 (769h5m42.065471023s) [noise=true , address=185.167.204.205:4035 , piece_id=HPHYG44QNKJYFSXEBH6FPYLFXLO6OUIVGDYL5GXJMP3PDNFUZEQA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.197860059 (769h5m42.068943094s) [noise=true , address=185.167.204.206:4064 , piece_id=JGCXE3QUORVKLRIWEXFZUCT4XUF3DRVA35ZKQU4DQODLZBOH5I4A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.827615690 (769h5m42.698698725s) [noise=true , address=81.171.14.70:4041    , piece_id=NLGMNQJDFISSAHBX3EC6WMWJLP74FMR4MRPUIX4GCCCQRG5RRRCQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.917676689 (769h5m42.788759725s) [noise=true , address=81.171.14.70:4034    , piece_id=5LGDXPCHMOQPZGJAEH3ZSPF64RSTVAJRQFZBXVWP5D3Y36YFLUUQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.932213350 (769h5m42.803296385s) [noise=true , address=81.171.14.70:4030    , piece_id=WV5BULPUQSAE7DUO2FWY74KF3MIHSQXYC673GNJ7XUBQXXM6KHEQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.956051121 (769h5m42.827134196s) [noise=true , address=81.171.14.70:4031    , piece_id=OJKJHNVWTPDITZC5CTQNB7UFPIVZDPE54ZXS4LWRM3XHYAB6443A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.005259852 (769h5m42.876342887s) [noise=true , address=185.167.204.205:4019 , piece_id=I7OP7AQS2DC3C7GZWG5RVJSLDKAZD2434WMBPRYMZFVAJYJG3JHQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.052708792 (769h5m42.923791837s) [noise=true , address=81.171.14.70:4046    , piece_id=OV3PMCD6UIFNSQNTCOS3GEQC4676KFJOUH5ZX576OFOYC5EXCZPQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.639749858 (769h5m43.510832903s) [noise=true , address=89.149.201.147:4004  , piece_id=NMPCYNSE3YVIAVDCWRO4VP6IN4DKZ2P4YIFYI6ZDTEU75QSFC4NQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.719270598 (769h5m43.590353643s) [noise=true , address=89.149.201.147:4019  , piece_id=LS3NXHELDI7H6VCTHO6LAGRDDG46ST4IF2XOFZDZ6EK756K5ACHQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.892052420 (769h5m43.763135466s) [noise=true , address=81.171.14.70:4035    , piece_id=BRXIJD7JX7L37MWMTVPL5KSESQUQEZNRXLBLL26XDMYIJSCRUD5A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.901787498 (769h5m43.772870533s) [noise=true , address=81.171.14.70:4039    , piece_id=VFXOSS5DRL43RDJGPLSH3667HWKAF5ZNC2CF5BA7TDR4NSCG75HA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.019887208 (769h5m43.890970243s) [noise=true , address=185.167.204.206:4050 , piece_id=KSCVNGXUGFV7XRNU5MFIKLLQGZ5BNHTIXQLVHWPMITZOQUA23DGQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.510337921 (769h5m44.381420957s) [noise=true , address=185.167.204.205:4028 , piece_id=QPB4EPPEENW277S4SBIOCWWMBLIE6GKCGEM3TV3MXN6SYFMQAI7Q, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.692198242 (769h5m44.563281277s) [noise=true , address=89.149.201.147:4010  , piece_id=C57ORVBIZE4QUWCVG3OVG3KW2Z2GYVWUXEOYG5R53KV2WUQLR7JQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.394574949 (769h5m46.265657984s) [noise=true , address=89.149.201.147:4014  , piece_id=KNV6OXYOC2H7LVZCW4ZWQRYRN5ZCMII3OO3FAS3YYQ22574KUCRQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.543864889 (769h5m46.414947934s) [noise=true , address=185.167.204.205:4014 , piece_id=XUCDQV4VW4ZZFTSZV2YEDAO5GQFP2ASW6BJFFC735UWEVIR3RNXA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.846238946 (769h5m46.717321982s) [noise=true , address=185.167.204.205:4006 , piece_id=W3XXEI7MG37VJHHK2CECNKIIHB7WSQABGVB5W5ZGAWSTFQ4NQMEQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.846258273 (769h5m46.717341308s) [seg_pos={0 0}, upload=672894]: Segment reached optimal threshold of 26 pieces.
Jan  6 16:34:46.846448986 (769h5m46.717532021s) [noise=true , address=185.167.204.206:4072 , piece_id=EJ4XCLNDZETIFIKE3FK2Q4NA35KT5LOTNGE253MKUCAIGH4FZD5A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v): protocol: expected piece hash; context canceled; EOF
Jan  6 16:34:46.846454998 (769h5m46.717538033s) [noise=true , address=185.167.204.206:4072 , piece_id=EJ4XCLNDZETIFIKE3FK2Q4NA35KT5LOTNGE253MKUCAIGH4FZD5A, seg_pos={0 0}, upload=672894]: Long tail cancellation due to optimal threshold
Jan  6 16:34:46.846579175 (769h5m46.71766221s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK): protocol: expected piece hash; context canceled; EOF
Jan  6 16:34:46.846595475 (769h5m46.71767851s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Long tail cancellation due to optimal threshold
Jan  6 16:34:46.846620453 (769h5m46.717703488s) [seg_pos={0 0}, upload=672894]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan  6 16:34:46.846729380 (769h5m46.717812415s) [upload=672894]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:34:46.846734600 (769h5m46.717817635s) [upload=672894]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:34:47.079572108 (769h5m46.950655143s) [upload=672894]: Done uploading segments...
Jan  6 16:34:47.079590854 (769h5m46.950673889s) [upload=672894]: Upload finished. err: <nil>
Duration: 21.828449875s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
57 out of 60 is failed (threshold=6) [XXXXXXXXXXXX.XX.XXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
5.001346258s
trace:
https://jaeger.dev.storj.io/trace/190db7a986616cf
FAIL (2026-01-06 16:42:43 UTC)
orbiter warning: context deadline exceeded
Duration: 5.001368811s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
22 out of 60 is failed (threshold=3) [X..X.XXXX..X.....XXX........X.......X..XX.....XX.X...X.XXX.X]
duration:
30.003293953s
trace:
https://jaeger.dev.storj.io/trace/5f4dda0b567de876
FAIL (2026-01-06 16:42:33 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Jan  6 16:42:04.336404604 (769h13m4.207487649s) [upload=672997]: Starting upload
Jan  6 16:42:04.336465750 (769h13m4.207548785s) [upload=672997]: Done starting upload
Jan  6 16:42:04.336584767 (769h13m4.207667803s) [upload=672997]: Uploading segments...
Jan  6 16:42:04.336689457 (769h13m4.207772502s) [upload=672997]: Got next segment. Inline: false
Jan  6 16:42:04.336746906 (769h13m4.207829951s) [upload=672997]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan  6 16:42:04.336753518 (769h13m4.207836553s) [upload=672997]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan  6 16:42:04.584660962 (769h13m4.455744007s) [seg_pos={0 0}, upload=672997]: Begin upload segment...
Jan  6 16:42:04.584704635 (769h13m4.45578768s) [seg_pos={0 0}, upload=672997]: Done begin upload segment.
Jan  6 16:42:04.584742537 (769h13m4.455825582s) [noise=true , address=45.91.116.12:4226    , piece_id=N44L2YIFUF6ZVDBLLSJNDUHZB4GJPFOWGDMLNQIHJRMBS5T7UISQ, seg_pos={0 0}, upload=672997]: Uploading piece...
Jan  6 16:42:04.584769058 (769h13m4.455852103s) [noise=true , address=145.79.227.7:4294    , piece_id=W7ZEAKEBYPDIWB4GBZ7U2F6JLHI6HUYQLUTCMR2DWHC7HZG6UCXA, seg_pos={0 0}, upload=672997]: Uploading piece...
Jan  6 16:42:04.585209086 (769h13m4.456292131s) [noise=true , address=145.79.227.13:4804   , piece_id=RRPAMKCN7BL3YTDM7YY2PXKYGY5SJY7IUTT7GESQ5DO5ATZL3FAA, seg_pos={0 0}, upload=672997]: Uploading piece...
Jan  6 16:42:04.585616492 (769h13m4.456699537s) [noise=true , address=145.79.227.48:4301   , piece_id=PLNIXTRXKLAISZFROULZ2WRO2OBB36GBQTNBDYULQU6FOOLXQPOA, seg_pos={0 0}, upload=672997]: Uploading piece...
Jan  6 16:42:04.586020841 (769h13m4.457103886s) [noise=true , address=145.79.227.17:4971   , piece_id=BP2OLPL7OMYN23IRZS7YGE3PYRBOCJ4EVXUJZ4Y7VTTJZSZHQWEA, seg_pos={0 0}, upload=672997]: Uploading piece...
Jan  6 16:42:04.586475066 (769h13m4.457558111s) [noise=true , address=45.91.116.14:4192    , piece_id=FNO36KFLST7FXA7FUS5ZG6FBCYAY5WLU7LOFDD42XH3CVDAMIERQ, seg_pos={0 0}, upload=672997]: Uploading piece...
Jan  6 16:42:04.586947777 (769h13m4.458030822s) [noise=true , address=145.79.227.25:4619   , piece_id=UDCGKOUHVYCW7Z46LZS35M4JODA6X6FDSYDJGDOZKC62MHNSFYYA, seg_pos={0 0}, upload=672997]: Uploading piece...
Jan  6 16:42:04.587305658 (769h13m4.458388703s) [noise=true , address=77.90.31.15:4700     , piece_id=7VOMJ5AGB7YMCLONC2AWE6V3GDQR6BFG54NO4UD2CUYEHMPBPMUQ, seg_pos={0 0}, upload=672997]: Uploading piece...
Jan  6 16:42:04.587704678 (769h13m4.458787723s) [noise=true , address=145.79.227.29:4941   , piece_id=37IYMTGYTBJSIILNMQOEQ5YCSGJPSMUY6F2D5X7X4I34K4J2XFRA, seg_pos={0 0}, upload=672997]: Uploading piece...
Jan  6 16:42:04.588875567 (769h13m4.459958602s) [noise=true , address=45.91.116.4:4939     , piece_id=KVFGDATMSOIGUNZZBTGWS7HU3EHFX7KK262C6LQEPGSTOUIBZYPA, seg_pos={0 0}, upload=672997]: Uploading piece...
Jan  6 16:42:04.588918449 (769h13m4.460001494s) [noise=true , address=145.79.227.45:4229   , piece_id=XQ2ONKJIVC4SKNW65TSU555E6U75MVHCALN6HFAPO2NGOK7HSPMQ, seg_pos={0 0}, upload=672997]: Uploading piece...
Jan  6 16:42:04.590215810 (769h13m4.461298845s) [noise=true , address=145.79.227.47:4267   , piece_id=JROEAM2QGBBDQ2OV3H4UPI3LPBOQS5DL7KTNV4G7QZW7HB4RP6JQ, seg_pos={0 0}, upload=672997]: Uploading piece...
Jan  6 16:42:04.590246849 (769h13m4.461329894s) [noise=true , address=145.79.227.18:4006   , piece_id=AKO7Q2EGI7E46ECVSYBWA47JNJD72ZX76ZAOMXFSZT2CXETFLROA, seg_pos={0 0}, upload=672997]: Uploading piece...
Jan  6 16:42:04.591532697 (769h13m4.462615742s) [noise=true , address=77.90.31.12:4589     , piece_id=4KSPWRQH7Y4HFNWKV46VVYMANY4N7MFRO364KIMRCTFXQDITTVIA, seg_pos={0 0}, upload=672997]: Uploading piece...
Jan  6 16:42:04.591708693 (769h13m4.462791738s) [noise=true , address=145.79.227.37:4566   , piece_id=STS3ZZRMEIBIBEZ4XONM566NR2QDSNKI6UJDL5SKIPWTYQ6WB4WQ, seg_pos={0 0}, upload=672997]: Uploading piece...
Jan  6 16:42:04.592932342 (769h13m4.464015388s) [noise=true , address=145.79.227.50:4373   , piece_id=WDDE4XRB5CXLPCAQSXIVCURLAITMQVZE5L4ULFTQELVWXM3CHVJA, seg_pos={0 0}, upload=672997]: Uploading piece...
Jan  6 16:42:06.280741511 (769h13m6.151824556s) [upload=672997]: Next returned nil segment
Jan  6 16:42:06.280756369 (769h13m6.151839404s) [upload=672997]: Waiting for error group managing segments...
Jan  6 16:42:06.575529914 (769h13m6.446612949s) [noise=true , address=145.79.227.7:4294    , piece_id=W7ZEAKEBYPDIWB4GBZ7U2F6JLHI6HUYQLUTCMR2DWHC7HZG6UCXA, seg_pos={0 0}, upload=672997]: Done uploading piece. err: <nil>
Jan  6 16:42:06.577423169 (769h13m6.448506204s) [noise=true , address=145.79.227.47:4267   , piece_id=JROEAM2QGBBDQ2OV3H4UPI3LPBOQS5DL7KTNV4G7QZW7HB4RP6JQ, seg_pos={0 0}, upload=672997]: Done uploading piece. err: <nil>
Jan  6 16:42:06.582994187 (769h13m6.454077222s) [noise=true , address=145.79.227.37:4566   , piece_id=STS3ZZRMEIBIBEZ4XONM566NR2QDSNKI6UJDL5SKIPWTYQ6WB4WQ, seg_pos={0 0}, upload=672997]: Done uploading piece. err: <nil>
Jan  6 16:42:06.585721019 (769h13m6.456804064s) [noise=true , address=145.79.227.25:4619   , piece_id=UDCGKOUHVYCW7Z46LZS35M4JODA6X6FDSYDJGDOZKC62MHNSFYYA, seg_pos={0 0}, upload=672997]: Done uploading piece. err: <nil>
Jan  6 16:42:06.590756638 (769h13m6.461839683s) [noise=true , address=145.79.227.50:4373   , piece_id=WDDE4XRB5CXLPCAQSXIVCURLAITMQVZE5L4ULFTQELVWXM3CHVJA, seg_pos={0 0}, upload=672997]: Done uploading piece. err: <nil>
Jan  6 16:42:06.601852616 (769h13m6.472935661s) [noise=true , address=145.79.227.13:4804   , piece_id=RRPAMKCN7BL3YTDM7YY2PXKYGY5SJY7IUTT7GESQ5DO5ATZL3FAA, seg_pos={0 0}, upload=672997]: Done uploading piece. err: <nil>
Jan  6 16:42:06.603383070 (769h13m6.474466115s) [noise=true , address=145.79.227.17:4971   , piece_id=BP2OLPL7OMYN23IRZS7YGE3PYRBOCJ4EVXUJZ4Y7VTTJZSZHQWEA, seg_pos={0 0}, upload=672997]: Done uploading piece. err: <nil>
Jan  6 16:42:06.605068148 (769h13m6.476151193s) [noise=true , address=145.79.227.45:4229   , piece_id=XQ2ONKJIVC4SKNW65TSU555E6U75MVHCALN6HFAPO2NGOK7HSPMQ, seg_pos={0 0}, upload=672997]: Done uploading piece. err: <nil>
Jan  6 16:42:06.610107685 (769h13m6.48119072s) [noise=true , address=145.79.227.18:4006   , piece_id=AKO7Q2EGI7E46ECVSYBWA47JNJD72ZX76ZAOMXFSZT2CXETFLROA, seg_pos={0 0}, upload=672997]: Done uploading piece. err: <nil>
Jan  6 16:42:07.018270440 (769h13m6.889353475s) [noise=true , address=77.90.31.15:4700     , piece_id=7VOMJ5AGB7YMCLONC2AWE6V3GDQR6BFG54NO4UD2CUYEHMPBPMUQ, seg_pos={0 0}, upload=672997]: Done uploading piece. err: <nil>
Jan  6 16:42:07.332993271 (769h13m7.204076306s) [noise=true , address=45.91.116.12:4226    , piece_id=N44L2YIFUF6ZVDBLLSJNDUHZB4GJPFOWGDMLNQIHJRMBS5T7UISQ, seg_pos={0 0}, upload=672997]: Done uploading piece. err: <nil>
Jan  6 16:42:07.340468745 (769h13m7.21155178s) [noise=true , address=45.91.116.14:4192    , piece_id=FNO36KFLST7FXA7FUS5ZG6FBCYAY5WLU7LOFDD42XH3CVDAMIERQ, seg_pos={0 0}, upload=672997]: Done uploading piece. err: <nil>
Jan  6 16:42:07.434496438 (769h13m7.305579473s) [noise=true , address=77.90.31.12:4589     , piece_id=4KSPWRQH7Y4HFNWKV46VVYMANY4N7MFRO364KIMRCTFXQDITTVIA, seg_pos={0 0}, upload=672997]: Done uploading piece. err: <nil>
Jan  6 16:42:07.434523580 (769h13m7.305606625s) [seg_pos={0 0}, upload=672997]: Segment reached optimal threshold of 13 pieces.
Jan  6 16:42:07.435106811 (769h13m7.306189856s) [noise=true , address=145.79.227.29:4941   , piece_id=37IYMTGYTBJSIILNMQOEQ5YCSGJPSMUY6F2D5X7X4I34K4J2XFRA, seg_pos={0 0}, upload=672997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125fn3wMJUNjsKTgsssLJkZAobCbaAS9jC2VyxXxbUA2ei4Gjni): protocol: expected piece hash; context canceled; EOF
Jan  6 16:42:07.435124604 (769h13m7.306207639s) [noise=true , address=145.79.227.29:4941   , piece_id=37IYMTGYTBJSIILNMQOEQ5YCSGJPSMUY6F2D5X7X4I34K4J2XFRA, seg_pos={0 0}, upload=672997]: Long tail cancellation due to optimal threshold
Jan  6 16:42:07.435512583 (769h13m7.306595628s) [noise=true , address=45.91.116.4:4939     , piece_id=KVFGDATMSOIGUNZZBTGWS7HU3EHFX7KK262C6LQEPGSTOUIBZYPA, seg_pos={0 0}, upload=672997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: read tcp 188.166.225.170:55082->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan  6 16:42:07.435519968 (769h13m7.306603013s) [noise=true , address=45.91.116.4:4939     , piece_id=KVFGDATMSOIGUNZZBTGWS7HU3EHFX7KK262C6LQEPGSTOUIBZYPA, seg_pos={0 0}, upload=672997]: Long tail cancellation due to optimal threshold
Jan  6 16:42:07.435705511 (769h13m7.306788556s) [noise=true , address=145.79.227.48:4301   , piece_id=PLNIXTRXKLAISZFROULZ2WRO2OBB36GBQTNBDYULQU6FOOLXQPOA, seg_pos={0 0}, upload=672997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18HkaSrdHHnvmeNz1ormfcSBZH8bMUgkX7X3JmGuqkmtBoak29): protocol: expected piece hash; context canceled; EOF
Jan  6 16:42:07.435711652 (769h13m7.306794687s) [noise=true , address=145.79.227.48:4301   , piece_id=PLNIXTRXKLAISZFROULZ2WRO2OBB36GBQTNBDYULQU6FOOLXQPOA, seg_pos={0 0}, upload=672997]: Long tail cancellation due to optimal threshold
Jan  6 16:42:07.435729918 (769h13m7.306812963s) [seg_pos={0 0}, upload=672997]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan  6 16:42:07.435808216 (769h13m7.306891251s) [upload=672997]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:42:07.435813586 (769h13m7.306896622s) [upload=672997]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:42:07.743911693 (769h13m7.614994738s) [upload=672997]: Done uploading segments...
Jan  6 16:42:07.743932522 (769h13m7.615015557s) [upload=672997]: Upload finished. err: <nil>
Duration: 30.003320093s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
3 out of 60 is failed (threshold=3) [X......................X......X.............................]
duration:
18.043821979s
trace:
https://jaeger.dev.storj.io/trace/d5e2d8b8127d900
FAIL (2026-01-06 16:42:32 UTC)
orbiter warning: context deadline exceeded
Duration: 18.043851275s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
13 out of 60 is failed (threshold=3) [.X......X..X.X.X........X.....XX......X..........XX...X.X...]
duration:
53.123568121s
trace:
https://jaeger.dev.storj.io/trace/76df23b0035fe317
FAIL (2026-01-06 16:42:30 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
g piece...
Jan  6 16:39:13.319055742 (769h10m13.190138787s) [noise=true , address=77.90.31.12:4616     , piece_id=JCWWA7PGIW23J4OKRJYW33S37FGDTIHBHTCLJSXXA4RH4SE7SWUA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.319538962 (769h10m13.190621997s) [noise=true , address=45.91.116.7:4014     , piece_id=65DKIVFKLQVVJEHZOT2P4KYDBHFD6URVGOFPFPABEGTOORENHXBA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.320135148 (769h10m13.191218193s) [noise=true , address=77.90.31.14:4668     , piece_id=QYCFWBXHW7BYLILL76W5MDFYQJKG6VZIZOYS3BF4OPDZLMJZXKVQ, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.320651351 (769h10m13.191734396s) [noise=true , address=145.79.227.11:4736   , piece_id=CTSU3X6OZJFWJSCLJBJ3EBGALWWGZL62ZJN436LKICHA4MLIQ2ZA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.321172283 (769h10m13.192255328s) [noise=true , address=64.49.13.27:4968     , piece_id=X4RAE7BUCM2R5NDMCCLH4TRGRCCXJBGUXZRE4QSNCFWNCJXLBSVA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.323456752 (769h10m13.194539787s) [noise=true , address=145.79.227.12:4773   , piece_id=JXMIF6EZKPD7EXZ646Z4XM6PG6I5P3EPTBJUWYQX4LMGAUFJT4EA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.325413879 (769h10m13.196496924s) [noise=true , address=145.79.227.33:4884   , piece_id=E5YFUCC75VTQBLZH3T5Q6VWEMRYAVOXNFKZQCPEP3ES64YCTXWPA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.327557260 (769h10m13.198640305s) [noise=true , address=145.79.227.6:4279    , piece_id=FNXBVE7QJDR67LLYLBXOCK3OQ7E6O5BIKKXPQMYOSMKIZSNK2EDQ, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.329566295 (769h10m13.20064934s) [noise=true , address=45.91.116.9:4282     , piece_id=GC72TML4SY7YTFQW4G3CVTOMGW5HWQLCWHBFQ3TKSZSUJUR252WA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.330791288 (769h10m13.201874323s) [noise=true , address=45.91.116.11:4139    , piece_id=NRCGKCNT7VWR4UUJE7HLXWT23K3BRUV2AHMDPOAVAXQF4UZKIPUA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.332349425 (769h10m13.20343246s) [noise=true , address=145.79.227.49:4371   , piece_id=YNFO36VNDUUPFQO4OFHIUEWUZWLYM2BUD7NF6VQEVWZHNCDSM6FA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.332771599 (769h10m13.203854634s) [noise=true , address=145.79.227.23:4557   , piece_id=QHDDTZ4DZAJCIU3I5HVA54JQLOHCCN73GSJAWZGC5JL6X4WV7ETA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.333853448 (769h10m13.204936494s) [noise=true , address=145.79.227.14:4843   , piece_id=GSANAVILLP6UTITEPDCBK5VHTTK7D2SBT7FQQI77H35I3SLNRVKA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.336038750 (769h10m13.207121785s) [noise=true , address=45.91.116.10:4107    , piece_id=I5OP5FXQBURDQG2LWEKGVZUBX7VV3CLD6Z2ZKKDK5UCX2RGFX3BA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:14.793189300 (769h10m14.664272335s) [upload=672957]: Next returned nil segment
Jan  6 16:39:14.793201272 (769h10m14.664284307s) [upload=672957]: Waiting for error group managing segments...
Jan  6 16:39:15.209058789 (769h10m15.080141834s) [noise=true , address=145.79.227.13:4835   , piece_id=MMTT3F5VEI2NUZ4T2QWJGNCPIB3O4EGLYANWMCFK6MC3YLQX3TXA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.251874898 (769h10m15.122957933s) [noise=true , address=145.79.227.14:4843   , piece_id=GSANAVILLP6UTITEPDCBK5VHTTK7D2SBT7FQQI77H35I3SLNRVKA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.299718689 (769h10m15.170801724s) [noise=true , address=145.79.227.11:4736   , piece_id=CTSU3X6OZJFWJSCLJBJ3EBGALWWGZL62ZJN436LKICHA4MLIQ2ZA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.351276473 (769h10m15.222359518s) [noise=true , address=145.79.227.12:4773   , piece_id=JXMIF6EZKPD7EXZ646Z4XM6PG6I5P3EPTBJUWYQX4LMGAUFJT4EA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.423449492 (769h10m15.294532527s) [noise=true , address=64.49.13.27:4968     , piece_id=X4RAE7BUCM2R5NDMCCLH4TRGRCCXJBGUXZRE4QSNCFWNCJXLBSVA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.580777038 (769h10m15.451860083s) [noise=true , address=77.90.31.14:4668     , piece_id=QYCFWBXHW7BYLILL76W5MDFYQJKG6VZIZOYS3BF4OPDZLMJZXKVQ, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.798468394 (769h10m15.66955143s) [noise=true , address=45.91.116.10:4107    , piece_id=I5OP5FXQBURDQG2LWEKGVZUBX7VV3CLD6Z2ZKKDK5UCX2RGFX3BA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.083877398 (769h10m15.954960443s) [noise=true , address=45.91.116.9:4282     , piece_id=GC72TML4SY7YTFQW4G3CVTOMGW5HWQLCWHBFQ3TKSZSUJUR252WA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.143050622 (769h10m16.014133687s) [noise=true , address=145.79.227.32:4865   , piece_id=Z5ZJEDUJG376CVJM4EDK6JPRJK2U5OR6RDD4YG5X4MXH4WKSZYSA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.194307403 (769h10m16.065390438s) [noise=true , address=145.79.227.33:4884   , piece_id=E5YFUCC75VTQBLZH3T5Q6VWEMRYAVOXNFKZQCPEP3ES64YCTXWPA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.244019653 (769h10m16.115102688s) [noise=true , address=145.79.227.49:4371   , piece_id=YNFO36VNDUUPFQO4OFHIUEWUZWLYM2BUD7NF6VQEVWZHNCDSM6FA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.587195683 (769h10m16.458278718s) [noise=true , address=45.91.116.7:4014     , piece_id=65DKIVFKLQVVJEHZOT2P4KYDBHFD6URVGOFPFPABEGTOORENHXBA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.752240113 (769h10m16.623323148s) [noise=true , address=145.79.227.21:4501   , piece_id=KQIHUVS3N56R5HFH2S2I6OE7PBZUXAHRGFMPK57KX7UA5I7BLCSQ, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.752279798 (769h10m16.623362843s) [noise=true , address=77.90.31.12:4616     , piece_id=JCWWA7PGIW23J4OKRJYW33S37FGDTIHBHTCLJSXXA4RH4SE7SWUA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.752291660 (769h10m16.623374695s) [seg_pos={0 0}, upload=672957]: Segment reached optimal threshold of 14 pieces.
Jan  6 16:39:16.752502462 (769h10m16.623585507s) [noise=true , address=145.79.227.6:4279    , piece_id=FNXBVE7QJDR67LLYLBXOCK3OQ7E6O5BIKKXPQMYOSMKIZSNK2EDQ, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uiA6tmZtmHDNaNtkhRtH31q4MJxn3C4bfrLAXmYz5mKmofHoG): protocol: write tcp 188.166.225.170:35010->145.79.227.6:4279: use of closed network connection; write tcp 188.166.225.170:35010->145.79.227.6:4279: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35010->145.79.227.6:4279: use of closed network connection
Jan  6 16:39:16.752510658 (769h10m16.623593693s) [noise=true , address=145.79.227.6:4279    , piece_id=FNXBVE7QJDR67LLYLBXOCK3OQ7E6O5BIKKXPQMYOSMKIZSNK2EDQ, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752633512 (769h10m16.623716557s) [noise=true , address=145.79.227.20:4454   , piece_id=W7E7EUDKE6BTN5I7TSTAMHETYKCXBU7RLYALUHELLOSTOAUFVB2Q, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SafkJPqKHLwMaqyZdLq3yJ1xdjqDtWHFrUwynmegUVwEC7y5J): protocol: expected piece hash; context canceled; EOF
Jan  6 16:39:16.752641587 (769h10m16.623724632s) [noise=true , address=145.79.227.20:4454   , piece_id=W7E7EUDKE6BTN5I7TSTAMHETYKCXBU7RLYALUHELLOSTOAUFVB2Q, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752764281 (769h10m16.623847316s) [noise=true , address=145.79.227.23:4557   , piece_id=QHDDTZ4DZAJCIU3I5HVA54JQLOHCCN73GSJAWZGC5JL6X4WV7ETA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sF7TCu1pL8mrtjoPULJ2VnZQHcJSMmvou2w7YgF58ZHQcnfJT): protocol: write tcp 188.166.225.170:43038->145.79.227.23:4557: use of closed network connection; write tcp 188.166.225.170:43038->145.79.227.23:4557: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43038->145.79.227.23:4557: use of closed network connection
Jan  6 16:39:16.752774249 (769h10m16.623857295s) [noise=true , address=145.79.227.23:4557   , piece_id=QHDDTZ4DZAJCIU3I5HVA54JQLOHCCN73GSJAWZGC5JL6X4WV7ETA, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752797775 (769h10m16.62388081s) [noise=true , address=45.91.116.11:4139    , piece_id=NRCGKCNT7VWR4UUJE7HLXWT23K3BRUV2AHMDPOAVAXQF4UZKIPUA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Tz3HEHACmHU9Gp2W2vHVuF5PXNmdsa5N8nYhwt52HCTjggqRD): protocol: read tcp 188.166.225.170:44870->45.91.116.11:4139: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan  6 16:39:16.752805439 (769h10m16.623888484s) [noise=true , address=45.91.116.11:4139    , piece_id=NRCGKCNT7VWR4UUJE7HLXWT23K3BRUV2AHMDPOAVAXQF4UZKIPUA, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752825017 (769h10m16.623908052s) [seg_pos={0 0}, upload=672957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jan  6 16:39:16.752902604 (769h10m16.623985659s) [upload=672957]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:39:16.752918193 (769h10m16.624001229s) [upload=672957]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:39:17.044336649 (769h10m16.915419684s) [upload=672957]: Done uploading segments...
Jan  6 16:39:17.044357138 (769h10m16.915440173s) [upload=672957]: Upload finished. err: <nil>
Duration: 53.123593408s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
57 out of 60 is failed (threshold=6) [XXXXXXXXXXXXX.XX.XXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
5.001196287s
trace:
https://jaeger.dev.storj.io/trace/2bb0733ddd0371a8
FAIL (2026-01-06 16:42:03 UTC)
orbiter warning: context deadline exceeded
Duration: 5.001218371s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
21 out of 60 is failed (threshold=3) [.X..X.XXXX..X.....XXX........X.......X..XX.....XX.X...X.XXX.]
duration:
11.451908805s
trace:
https://jaeger.dev.storj.io/trace/290c1e2e04f13713
FAIL (2026-01-06 16:41:57 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Jan  6 16:41:02.198243973 (769h12m2.069327018s) [upload=672981]: Starting upload
Jan  6 16:41:02.198327172 (769h12m2.069410217s) [upload=672981]: Done starting upload
Jan  6 16:41:02.198489230 (769h12m2.069572275s) [upload=672981]: Uploading segments...
Jan  6 16:41:02.198604891 (769h12m2.069687936s) [upload=672981]: Got next segment. Inline: false
Jan  6 16:41:02.198707155 (769h12m2.0697902s) [upload=672981]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan  6 16:41:02.198716072 (769h12m2.069799107s) [upload=672981]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan  6 16:41:02.443306500 (769h12m2.314389545s) [seg_pos={0 0}, upload=672981]: Begin upload segment...
Jan  6 16:41:02.443364520 (769h12m2.314447565s) [seg_pos={0 0}, upload=672981]: Done begin upload segment.
Jan  6 16:41:02.443410819 (769h12m2.314493864s) [noise=true , address=45.91.116.7:4010     , piece_id=BOM7SIKTWRF7XEYQJCWPFKFKGZWOOBW3LF5GOW23H36VRNUUFDTA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.443421479 (769h12m2.314504524s) [noise=true , address=45.91.116.10:4081    , piece_id=GTCPJG56ECKSRNVN2ALOA4YGJC3VX52DSNPTRBJCAMV6V5LQPJMQ, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.443544203 (769h12m2.314627248s) [noise=true , address=45.91.116.14:4189    , piece_id=JEODWVNZFW34BJDG5N7WFV7BZEEFRGSI35YGUELKCH4A7JZJL7WQ, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.444064463 (769h12m2.315147508s) [noise=true , address=145.79.227.13:4802   , piece_id=4GYCIUUEVQMBJYKH2ZZNIMHO72WKFYYAUV54D6R57F7RQUT24INA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.444517265 (769h12m2.3156003s) [noise=true , address=145.79.227.14:4838   , piece_id=WIFW46L4VJZZCJDNHTWWRCS24OEZ32OBJWUVEZBIPGSUOZVCCHTQ, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.444865548 (769h12m2.315948583s) [noise=true , address=45.91.116.13:4154    , piece_id=HMBRWLBDPWSZNGCD74IHJHYRNQNH3ZAIFE56SXSS7JBTBX3S3ZQA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.445289355 (769h12m2.3163724s) [noise=true , address=145.79.227.5:4221    , piece_id=5RH4NTNPYEOOXOPB6QI3D4CFZU5VGXH2EBW6FS75R6CXMGI7B7FA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.445680250 (769h12m2.316763295s) [noise=true , address=145.79.227.15:4876   , piece_id=KYIIR6MP5GCFYW6UNSNWHYYHKLXEDQIXG557OUUF2D42T4QNQNSA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.446139464 (769h12m2.317222499s) [noise=true , address=45.91.116.9:4263     , piece_id=FDNSX3Y6F225QDXPRKEKITLXEBLMAY4XNAEDHDI4PJWOHKNNY5JQ, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.448097182 (769h12m2.319180217s) [noise=true , address=145.79.227.17:4971   , piece_id=5QGHTV7VGEMUC4BNBCUZFBDT7W6OFJSN6YY2DHAVMSCILDJN56AQ, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.449387619 (769h12m2.320470664s) [noise=true , address=145.79.227.48:4302   , piece_id=6BXCIVEPUY2HHNXTEYMBIBV5KW56RJXZGJTHSYBCLZU2BJBXHFCQ, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.451356228 (769h12m2.322439263s) [noise=true , address=145.79.227.11:4729   , piece_id=CWH3NMBS4LNSR3YZU3UOG5HV2GIMEYTGATJ4N27I6F7P3YA3QSDA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.453023954 (769h12m2.324106989s) [noise=true , address=145.79.227.24:4583   , piece_id=7VYSH5AW65PLJKHQHOXIDJ635YEO3NYMXBVLTAHQZPIVF3TW6AJA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.454829421 (769h12m2.325912466s) [noise=true , address=145.79.227.33:4870   , piece_id=PNTUSRLHBEDGKOHUS2PGDHYUYZ3O6SFCI7245DJOSHV3IIJIMIYA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.457973478 (769h12m2.329056523s) [noise=true , address=145.79.227.42:4121   , piece_id=MJLWPAYJBU4DVSTMOZSXHCNGS7FBE43HXMDWDLL3OOQ6XFUNXKWA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.459371360 (769h12m2.330454405s) [noise=true , address=145.79.227.46:4601   , piece_id=3S4TXXQHRQBCLEJB4WAS5D3CS2VCMKACRLUC52AZGXUD5NF5K46Q, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:04.156938259 (769h12m4.028021304s) [upload=672981]: Next returned nil segment
Jan  6 16:41:04.156976411 (769h12m4.028059457s) [upload=672981]: Waiting for error group managing segments...
Jan  6 16:41:04.446586444 (769h12m4.317669479s) [noise=true , address=145.79.227.13:4802   , piece_id=4GYCIUUEVQMBJYKH2ZZNIMHO72WKFYYAUV54D6R57F7RQUT24INA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.456220869 (769h12m4.327303914s) [noise=true , address=145.79.227.14:4838   , piece_id=WIFW46L4VJZZCJDNHTWWRCS24OEZ32OBJWUVEZBIPGSUOZVCCHTQ, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.462811298 (769h12m4.333894333s) [noise=true , address=145.79.227.46:4601   , piece_id=3S4TXXQHRQBCLEJB4WAS5D3CS2VCMKACRLUC52AZGXUD5NF5K46Q, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.470015607 (769h12m4.341098652s) [noise=true , address=145.79.227.42:4121   , piece_id=MJLWPAYJBU4DVSTMOZSXHCNGS7FBE43HXMDWDLL3OOQ6XFUNXKWA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.479755061 (769h12m4.350838096s) [noise=true , address=145.79.227.17:4971   , piece_id=5QGHTV7VGEMUC4BNBCUZFBDT7W6OFJSN6YY2DHAVMSCILDJN56AQ, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.518407642 (769h12m4.389490687s) [noise=true , address=145.79.227.11:4729   , piece_id=CWH3NMBS4LNSR3YZU3UOG5HV2GIMEYTGATJ4N27I6F7P3YA3QSDA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.530405669 (769h12m4.401488714s) [noise=true , address=145.79.227.24:4583   , piece_id=7VYSH5AW65PLJKHQHOXIDJ635YEO3NYMXBVLTAHQZPIVF3TW6AJA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.566796432 (769h12m4.437879467s) [noise=true , address=145.79.227.5:4221    , piece_id=5RH4NTNPYEOOXOPB6QI3D4CFZU5VGXH2EBW6FS75R6CXMGI7B7FA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.581701784 (769h12m4.452784829s) [noise=true , address=145.79.227.15:4876   , piece_id=KYIIR6MP5GCFYW6UNSNWHYYHKLXEDQIXG557OUUF2D42T4QNQNSA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.918193646 (769h12m4.789276711s) [noise=true , address=45.91.116.10:4081    , piece_id=GTCPJG56ECKSRNVN2ALOA4YGJC3VX52DSNPTRBJCAMV6V5LQPJMQ, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:05.302753818 (769h12m5.173836863s) [noise=true , address=45.91.116.14:4189    , piece_id=JEODWVNZFW34BJDG5N7WFV7BZEEFRGSI35YGUELKCH4A7JZJL7WQ, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:05.646462512 (769h12m5.517545557s) [noise=true , address=45.91.116.13:4154    , piece_id=HMBRWLBDPWSZNGCD74IHJHYRNQNH3ZAIFE56SXSS7JBTBX3S3ZQA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:06.027302039 (769h12m5.898385074s) [noise=true , address=145.79.227.48:4302   , piece_id=6BXCIVEPUY2HHNXTEYMBIBV5KW56RJXZGJTHSYBCLZU2BJBXHFCQ, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:06.027333479 (769h12m5.898416514s) [seg_pos={0 0}, upload=672981]: Segment reached optimal threshold of 13 pieces.
Jan  6 16:41:06.027609695 (769h12m5.89869274s) [noise=true , address=45.91.116.7:4010     , piece_id=BOM7SIKTWRF7XEYQJCWPFKFKGZWOOBW3LF5GOW23H36VRNUUFDTA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jan  6 16:41:06.027619053 (769h12m5.898702108s) [noise=true , address=45.91.116.7:4010     , piece_id=BOM7SIKTWRF7XEYQJCWPFKFKGZWOOBW3LF5GOW23H36VRNUUFDTA, seg_pos={0 0}, upload=672981]: Long tail cancellation due to optimal threshold
Jan  6 16:41:06.027687904 (769h12m5.898770939s) [noise=true , address=45.91.116.9:4263     , piece_id=FDNSX3Y6F225QDXPRKEKITLXEBLMAY4XNAEDHDI4PJWOHKNNY5JQ, seg_pos={0 0}, upload=672981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF
Jan  6 16:41:06.027697482 (769h12m5.898780527s) [noise=true , address=45.91.116.9:4263     , piece_id=FDNSX3Y6F225QDXPRKEKITLXEBLMAY4XNAEDHDI4PJWOHKNNY5JQ, seg_pos={0 0}, upload=672981]: Long tail cancellation due to optimal threshold
Jan  6 16:41:06.027908494 (769h12m5.89899157s) [noise=true , address=145.79.227.33:4870   , piece_id=PNTUSRLHBEDGKOHUS2PGDHYUYZ3O6SFCI7245DJOSHV3IIJIMIYA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sDH7ccMQhjePqpbTkkxMAtumNLxpMHiT56cnfaYrUTFYxuPVt): protocol: expected piece hash; context canceled; EOF
Jan  6 16:41:06.027918603 (769h12m5.899001648s) [noise=true , address=145.79.227.33:4870   , piece_id=PNTUSRLHBEDGKOHUS2PGDHYUYZ3O6SFCI7245DJOSHV3IIJIMIYA, seg_pos={0 0}, upload=672981]: Long tail cancellation due to optimal threshold
Jan  6 16:41:06.027942469 (769h12m5.899025504s) [seg_pos={0 0}, upload=672981]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan  6 16:41:06.028040946 (769h12m5.899123981s) [upload=672981]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:41:06.028048010 (769h12m5.899131045s) [upload=672981]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:41:06.314366555 (769h12m6.1854496s) [upload=672981]: Done uploading segments...
Jan  6 16:41:06.314434635 (769h12m6.18551768s) [upload=672981]: Upload finished. err: <nil>
Duration: 11.451934863s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
13 out of 60 is failed (threshold=3) [XX......................X.X....X.X.X....X.X...X....XXX......]
duration:
21.780743332s
trace:
https://jaeger.dev.storj.io/trace/73dd0c7413e4b545
FAIL (2026-01-06 16:41:47 UTC)
orbiter warning: context deadline exceeded
Duration: 21.780773438s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)
antiblip:
2 out of 60 is failed (threshold=3) [...................................................XX.......]
duration:
20.354832618s
trace:
https://jaeger.dev.storj.io/trace/4b3b841c440bfc62
FAIL (2026-01-06 16:41:35 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
JXEZNOOX75ENDV6EBXU4NR4HW4GTZMVA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913312874 (769h5m40.784395909s) [noise=true , address=81.171.14.70:4035    , piece_id=BRXIJD7JX7L37MWMTVPL5KSESQUQEZNRXLBLL26XDMYIJSCRUD5A, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913698277 (769h5m40.784781322s) [noise=true , address=81.171.14.70:4030    , piece_id=WV5BULPUQSAE7DUO2FWY74KF3MIHSQXYC673GNJ7XUBQXXM6KHEQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913862180 (769h5m40.784945215s) [noise=true , address=89.149.201.147:4004  , piece_id=NMPCYNSE3YVIAVDCWRO4VP6IN4DKZ2P4YIFYI6ZDTEU75QSFC4NQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.914346662 (769h5m40.785429707s) [noise=true , address=185.167.204.205:4035 , piece_id=HPHYG44QNKJYFSXEBH6FPYLFXLO6OUIVGDYL5GXJMP3PDNFUZEQA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.914842456 (769h5m40.785925491s) [noise=true , address=81.171.14.70:4034    , piece_id=5LGDXPCHMOQPZGJAEH3ZSPF64RSTVAJRQFZBXVWP5D3Y36YFLUUQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.915307461 (769h5m40.786390506s) [noise=true , address=89.149.201.147:4019  , piece_id=LS3NXHELDI7H6VCTHO6LAGRDDG46ST4IF2XOFZDZ6EK756K5ACHQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.915793817 (769h5m40.786876863s) [noise=true , address=185.167.204.206:4067 , piece_id=GLITDKTEKYV4CFGL3TJXWJDFOBYPFX2UTNJFPUCZDJM3HOWVFS4Q, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.916279973 (769h5m40.787363018s) [noise=true , address=89.149.201.147:4014  , piece_id=KNV6OXYOC2H7LVZCW4ZWQRYRN5ZCMII3OO3FAS3YYQ22574KUCRQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.916759336 (769h5m40.787842371s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.920089677 (769h5m40.791172722s) [noise=true , address=185.167.204.205:4014 , piece_id=XUCDQV4VW4ZZFTSZV2YEDAO5GQFP2ASW6BJFFC735UWEVIR3RNXA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:41.741185400 (769h5m41.612268445s) [upload=672894]: Next returned nil segment
Jan  6 16:34:41.741201020 (769h5m41.612284055s) [upload=672894]: Waiting for error group managing segments...
Jan  6 16:34:42.136024704 (769h5m42.00710774s) [noise=true , address=185.167.204.205:4036 , piece_id=4XKAIZPEJZAVAS7P2QILKY7XIDIIRIRWH7HTN5OVXUHAL2Y3H4SQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.137413950 (769h5m42.008496995s) [noise=true , address=185.167.204.206:4051 , piece_id=XWKJM6ZIHISLSESJ2ZQQQMZTGMK5MHH6L4ZD5RXJ3SMAVASVVVOA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.137527506 (769h5m42.008610541s) [noise=true , address=185.167.204.206:4049 , piece_id=4T665GS7XTBVEHUVT6SFIQW4F3TTBPDKZ57PKQHHHWWKC4P4JQPQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.138183485 (769h5m42.00926653s) [noise=true , address=185.167.204.206:4048 , piece_id=ZGQYUYRK5PK24UF4PYT67FHNPR4GVFDCJ2OEGRFENLB2GYR2LFVQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.139902609 (769h5m42.010985644s) [noise=true , address=185.167.204.206:4056 , piece_id=KYHDW5RLJTZCURXCFIP7HN3ZYLPPHK7FTO6IIGQ5VJJUPF4ICQXA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.140277843 (769h5m42.011360878s) [noise=true , address=185.167.204.206:4067 , piece_id=GLITDKTEKYV4CFGL3TJXWJDFOBYPFX2UTNJFPUCZDJM3HOWVFS4Q, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.142394303 (769h5m42.013477348s) [noise=true , address=185.167.204.205:4021 , piece_id=U4X3NZRT7MHG2JTNRVDRYIC72UDMHUQUCY4VFW7JFVLLVACZN7OA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.179529514 (769h5m42.050612549s) [noise=true , address=185.167.204.206:4053 , piece_id=HBFCBJCOLPJLNE4INHP4JXEZNOOX75ENDV6EBXU4NR4HW4GTZMVA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.194387988 (769h5m42.065471023s) [noise=true , address=185.167.204.205:4035 , piece_id=HPHYG44QNKJYFSXEBH6FPYLFXLO6OUIVGDYL5GXJMP3PDNFUZEQA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.197860059 (769h5m42.068943094s) [noise=true , address=185.167.204.206:4064 , piece_id=JGCXE3QUORVKLRIWEXFZUCT4XUF3DRVA35ZKQU4DQODLZBOH5I4A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.827615690 (769h5m42.698698725s) [noise=true , address=81.171.14.70:4041    , piece_id=NLGMNQJDFISSAHBX3EC6WMWJLP74FMR4MRPUIX4GCCCQRG5RRRCQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.917676689 (769h5m42.788759725s) [noise=true , address=81.171.14.70:4034    , piece_id=5LGDXPCHMOQPZGJAEH3ZSPF64RSTVAJRQFZBXVWP5D3Y36YFLUUQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.932213350 (769h5m42.803296385s) [noise=true , address=81.171.14.70:4030    , piece_id=WV5BULPUQSAE7DUO2FWY74KF3MIHSQXYC673GNJ7XUBQXXM6KHEQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.956051121 (769h5m42.827134196s) [noise=true , address=81.171.14.70:4031    , piece_id=OJKJHNVWTPDITZC5CTQNB7UFPIVZDPE54ZXS4LWRM3XHYAB6443A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.005259852 (769h5m42.876342887s) [noise=true , address=185.167.204.205:4019 , piece_id=I7OP7AQS2DC3C7GZWG5RVJSLDKAZD2434WMBPRYMZFVAJYJG3JHQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.052708792 (769h5m42.923791837s) [noise=true , address=81.171.14.70:4046    , piece_id=OV3PMCD6UIFNSQNTCOS3GEQC4676KFJOUH5ZX576OFOYC5EXCZPQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.639749858 (769h5m43.510832903s) [noise=true , address=89.149.201.147:4004  , piece_id=NMPCYNSE3YVIAVDCWRO4VP6IN4DKZ2P4YIFYI6ZDTEU75QSFC4NQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.719270598 (769h5m43.590353643s) [noise=true , address=89.149.201.147:4019  , piece_id=LS3NXHELDI7H6VCTHO6LAGRDDG46ST4IF2XOFZDZ6EK756K5ACHQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.892052420 (769h5m43.763135466s) [noise=true , address=81.171.14.70:4035    , piece_id=BRXIJD7JX7L37MWMTVPL5KSESQUQEZNRXLBLL26XDMYIJSCRUD5A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.901787498 (769h5m43.772870533s) [noise=true , address=81.171.14.70:4039    , piece_id=VFXOSS5DRL43RDJGPLSH3667HWKAF5ZNC2CF5BA7TDR4NSCG75HA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.019887208 (769h5m43.890970243s) [noise=true , address=185.167.204.206:4050 , piece_id=KSCVNGXUGFV7XRNU5MFIKLLQGZ5BNHTIXQLVHWPMITZOQUA23DGQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.510337921 (769h5m44.381420957s) [noise=true , address=185.167.204.205:4028 , piece_id=QPB4EPPEENW277S4SBIOCWWMBLIE6GKCGEM3TV3MXN6SYFMQAI7Q, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.692198242 (769h5m44.563281277s) [noise=true , address=89.149.201.147:4010  , piece_id=C57ORVBIZE4QUWCVG3OVG3KW2Z2GYVWUXEOYG5R53KV2WUQLR7JQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.394574949 (769h5m46.265657984s) [noise=true , address=89.149.201.147:4014  , piece_id=KNV6OXYOC2H7LVZCW4ZWQRYRN5ZCMII3OO3FAS3YYQ22574KUCRQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.543864889 (769h5m46.414947934s) [noise=true , address=185.167.204.205:4014 , piece_id=XUCDQV4VW4ZZFTSZV2YEDAO5GQFP2ASW6BJFFC735UWEVIR3RNXA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.846238946 (769h5m46.717321982s) [noise=true , address=185.167.204.205:4006 , piece_id=W3XXEI7MG37VJHHK2CECNKIIHB7WSQABGVB5W5ZGAWSTFQ4NQMEQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.846258273 (769h5m46.717341308s) [seg_pos={0 0}, upload=672894]: Segment reached optimal threshold of 26 pieces.
Jan  6 16:34:46.846448986 (769h5m46.717532021s) [noise=true , address=185.167.204.206:4072 , piece_id=EJ4XCLNDZETIFIKE3FK2Q4NA35KT5LOTNGE253MKUCAIGH4FZD5A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v): protocol: expected piece hash; context canceled; EOF
Jan  6 16:34:46.846454998 (769h5m46.717538033s) [noise=true , address=185.167.204.206:4072 , piece_id=EJ4XCLNDZETIFIKE3FK2Q4NA35KT5LOTNGE253MKUCAIGH4FZD5A, seg_pos={0 0}, upload=672894]: Long tail cancellation due to optimal threshold
Jan  6 16:34:46.846579175 (769h5m46.71766221s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK): protocol: expected piece hash; context canceled; EOF
Jan  6 16:34:46.846595475 (769h5m46.71767851s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Long tail cancellation due to optimal threshold
Jan  6 16:34:46.846620453 (769h5m46.717703488s) [seg_pos={0 0}, upload=672894]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan  6 16:34:46.846729380 (769h5m46.717812415s) [upload=672894]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:34:46.846734600 (769h5m46.717817635s) [upload=672894]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:34:47.079572108 (769h5m46.950655143s) [upload=672894]: Done uploading segments...
Jan  6 16:34:47.079590854 (769h5m46.950673889s) [upload=672894]: Upload finished. err: <nil>
Duration: 20.354863046s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
3 out of 60 is failed (threshold=3) [.X......................X......X............................]
duration:
16.199378446s
trace:
https://jaeger.dev.storj.io/trace/d5e2d8b8127d900
FAIL (2026-01-06 16:41:34 UTC)
orbiter warning: context deadline exceeded
Duration: 16.199408712s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
21 out of 60 is failed (threshold=3) [..X..X.XXXX..X.....XXX........X.......X..XX.....XX.X...X.XXX]
duration:
30.003917285s
trace:
https://jaeger.dev.storj.io/trace/290c1e2e04f13713
FAIL (2026-01-06 16:41:31 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Jan  6 16:41:02.198243973 (769h12m2.069327018s) [upload=672981]: Starting upload
Jan  6 16:41:02.198327172 (769h12m2.069410217s) [upload=672981]: Done starting upload
Jan  6 16:41:02.198489230 (769h12m2.069572275s) [upload=672981]: Uploading segments...
Jan  6 16:41:02.198604891 (769h12m2.069687936s) [upload=672981]: Got next segment. Inline: false
Jan  6 16:41:02.198707155 (769h12m2.0697902s) [upload=672981]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan  6 16:41:02.198716072 (769h12m2.069799107s) [upload=672981]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan  6 16:41:02.443306500 (769h12m2.314389545s) [seg_pos={0 0}, upload=672981]: Begin upload segment...
Jan  6 16:41:02.443364520 (769h12m2.314447565s) [seg_pos={0 0}, upload=672981]: Done begin upload segment.
Jan  6 16:41:02.443410819 (769h12m2.314493864s) [noise=true , address=45.91.116.7:4010     , piece_id=BOM7SIKTWRF7XEYQJCWPFKFKGZWOOBW3LF5GOW23H36VRNUUFDTA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.443421479 (769h12m2.314504524s) [noise=true , address=45.91.116.10:4081    , piece_id=GTCPJG56ECKSRNVN2ALOA4YGJC3VX52DSNPTRBJCAMV6V5LQPJMQ, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.443544203 (769h12m2.314627248s) [noise=true , address=45.91.116.14:4189    , piece_id=JEODWVNZFW34BJDG5N7WFV7BZEEFRGSI35YGUELKCH4A7JZJL7WQ, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.444064463 (769h12m2.315147508s) [noise=true , address=145.79.227.13:4802   , piece_id=4GYCIUUEVQMBJYKH2ZZNIMHO72WKFYYAUV54D6R57F7RQUT24INA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.444517265 (769h12m2.3156003s) [noise=true , address=145.79.227.14:4838   , piece_id=WIFW46L4VJZZCJDNHTWWRCS24OEZ32OBJWUVEZBIPGSUOZVCCHTQ, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.444865548 (769h12m2.315948583s) [noise=true , address=45.91.116.13:4154    , piece_id=HMBRWLBDPWSZNGCD74IHJHYRNQNH3ZAIFE56SXSS7JBTBX3S3ZQA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.445289355 (769h12m2.3163724s) [noise=true , address=145.79.227.5:4221    , piece_id=5RH4NTNPYEOOXOPB6QI3D4CFZU5VGXH2EBW6FS75R6CXMGI7B7FA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.445680250 (769h12m2.316763295s) [noise=true , address=145.79.227.15:4876   , piece_id=KYIIR6MP5GCFYW6UNSNWHYYHKLXEDQIXG557OUUF2D42T4QNQNSA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.446139464 (769h12m2.317222499s) [noise=true , address=45.91.116.9:4263     , piece_id=FDNSX3Y6F225QDXPRKEKITLXEBLMAY4XNAEDHDI4PJWOHKNNY5JQ, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.448097182 (769h12m2.319180217s) [noise=true , address=145.79.227.17:4971   , piece_id=5QGHTV7VGEMUC4BNBCUZFBDT7W6OFJSN6YY2DHAVMSCILDJN56AQ, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.449387619 (769h12m2.320470664s) [noise=true , address=145.79.227.48:4302   , piece_id=6BXCIVEPUY2HHNXTEYMBIBV5KW56RJXZGJTHSYBCLZU2BJBXHFCQ, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.451356228 (769h12m2.322439263s) [noise=true , address=145.79.227.11:4729   , piece_id=CWH3NMBS4LNSR3YZU3UOG5HV2GIMEYTGATJ4N27I6F7P3YA3QSDA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.453023954 (769h12m2.324106989s) [noise=true , address=145.79.227.24:4583   , piece_id=7VYSH5AW65PLJKHQHOXIDJ635YEO3NYMXBVLTAHQZPIVF3TW6AJA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.454829421 (769h12m2.325912466s) [noise=true , address=145.79.227.33:4870   , piece_id=PNTUSRLHBEDGKOHUS2PGDHYUYZ3O6SFCI7245DJOSHV3IIJIMIYA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.457973478 (769h12m2.329056523s) [noise=true , address=145.79.227.42:4121   , piece_id=MJLWPAYJBU4DVSTMOZSXHCNGS7FBE43HXMDWDLL3OOQ6XFUNXKWA, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:02.459371360 (769h12m2.330454405s) [noise=true , address=145.79.227.46:4601   , piece_id=3S4TXXQHRQBCLEJB4WAS5D3CS2VCMKACRLUC52AZGXUD5NF5K46Q, seg_pos={0 0}, upload=672981]: Uploading piece...
Jan  6 16:41:04.156938259 (769h12m4.028021304s) [upload=672981]: Next returned nil segment
Jan  6 16:41:04.156976411 (769h12m4.028059457s) [upload=672981]: Waiting for error group managing segments...
Jan  6 16:41:04.446586444 (769h12m4.317669479s) [noise=true , address=145.79.227.13:4802   , piece_id=4GYCIUUEVQMBJYKH2ZZNIMHO72WKFYYAUV54D6R57F7RQUT24INA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.456220869 (769h12m4.327303914s) [noise=true , address=145.79.227.14:4838   , piece_id=WIFW46L4VJZZCJDNHTWWRCS24OEZ32OBJWUVEZBIPGSUOZVCCHTQ, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.462811298 (769h12m4.333894333s) [noise=true , address=145.79.227.46:4601   , piece_id=3S4TXXQHRQBCLEJB4WAS5D3CS2VCMKACRLUC52AZGXUD5NF5K46Q, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.470015607 (769h12m4.341098652s) [noise=true , address=145.79.227.42:4121   , piece_id=MJLWPAYJBU4DVSTMOZSXHCNGS7FBE43HXMDWDLL3OOQ6XFUNXKWA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.479755061 (769h12m4.350838096s) [noise=true , address=145.79.227.17:4971   , piece_id=5QGHTV7VGEMUC4BNBCUZFBDT7W6OFJSN6YY2DHAVMSCILDJN56AQ, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.518407642 (769h12m4.389490687s) [noise=true , address=145.79.227.11:4729   , piece_id=CWH3NMBS4LNSR3YZU3UOG5HV2GIMEYTGATJ4N27I6F7P3YA3QSDA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.530405669 (769h12m4.401488714s) [noise=true , address=145.79.227.24:4583   , piece_id=7VYSH5AW65PLJKHQHOXIDJ635YEO3NYMXBVLTAHQZPIVF3TW6AJA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.566796432 (769h12m4.437879467s) [noise=true , address=145.79.227.5:4221    , piece_id=5RH4NTNPYEOOXOPB6QI3D4CFZU5VGXH2EBW6FS75R6CXMGI7B7FA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.581701784 (769h12m4.452784829s) [noise=true , address=145.79.227.15:4876   , piece_id=KYIIR6MP5GCFYW6UNSNWHYYHKLXEDQIXG557OUUF2D42T4QNQNSA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:04.918193646 (769h12m4.789276711s) [noise=true , address=45.91.116.10:4081    , piece_id=GTCPJG56ECKSRNVN2ALOA4YGJC3VX52DSNPTRBJCAMV6V5LQPJMQ, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:05.302753818 (769h12m5.173836863s) [noise=true , address=45.91.116.14:4189    , piece_id=JEODWVNZFW34BJDG5N7WFV7BZEEFRGSI35YGUELKCH4A7JZJL7WQ, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:05.646462512 (769h12m5.517545557s) [noise=true , address=45.91.116.13:4154    , piece_id=HMBRWLBDPWSZNGCD74IHJHYRNQNH3ZAIFE56SXSS7JBTBX3S3ZQA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:06.027302039 (769h12m5.898385074s) [noise=true , address=145.79.227.48:4302   , piece_id=6BXCIVEPUY2HHNXTEYMBIBV5KW56RJXZGJTHSYBCLZU2BJBXHFCQ, seg_pos={0 0}, upload=672981]: Done uploading piece. err: <nil>
Jan  6 16:41:06.027333479 (769h12m5.898416514s) [seg_pos={0 0}, upload=672981]: Segment reached optimal threshold of 13 pieces.
Jan  6 16:41:06.027609695 (769h12m5.89869274s) [noise=true , address=45.91.116.7:4010     , piece_id=BOM7SIKTWRF7XEYQJCWPFKFKGZWOOBW3LF5GOW23H36VRNUUFDTA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF
Jan  6 16:41:06.027619053 (769h12m5.898702108s) [noise=true , address=45.91.116.7:4010     , piece_id=BOM7SIKTWRF7XEYQJCWPFKFKGZWOOBW3LF5GOW23H36VRNUUFDTA, seg_pos={0 0}, upload=672981]: Long tail cancellation due to optimal threshold
Jan  6 16:41:06.027687904 (769h12m5.898770939s) [noise=true , address=45.91.116.9:4263     , piece_id=FDNSX3Y6F225QDXPRKEKITLXEBLMAY4XNAEDHDI4PJWOHKNNY5JQ, seg_pos={0 0}, upload=672981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF
Jan  6 16:41:06.027697482 (769h12m5.898780527s) [noise=true , address=45.91.116.9:4263     , piece_id=FDNSX3Y6F225QDXPRKEKITLXEBLMAY4XNAEDHDI4PJWOHKNNY5JQ, seg_pos={0 0}, upload=672981]: Long tail cancellation due to optimal threshold
Jan  6 16:41:06.027908494 (769h12m5.89899157s) [noise=true , address=145.79.227.33:4870   , piece_id=PNTUSRLHBEDGKOHUS2PGDHYUYZ3O6SFCI7245DJOSHV3IIJIMIYA, seg_pos={0 0}, upload=672981]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sDH7ccMQhjePqpbTkkxMAtumNLxpMHiT56cnfaYrUTFYxuPVt): protocol: expected piece hash; context canceled; EOF
Jan  6 16:41:06.027918603 (769h12m5.899001648s) [noise=true , address=145.79.227.33:4870   , piece_id=PNTUSRLHBEDGKOHUS2PGDHYUYZ3O6SFCI7245DJOSHV3IIJIMIYA, seg_pos={0 0}, upload=672981]: Long tail cancellation due to optimal threshold
Jan  6 16:41:06.027942469 (769h12m5.899025504s) [seg_pos={0 0}, upload=672981]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jan  6 16:41:06.028040946 (769h12m5.899123981s) [upload=672981]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:41:06.028048010 (769h12m5.899131045s) [upload=672981]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:41:06.314366555 (769h12m6.1854496s) [upload=672981]: Done uploading segments...
Jan  6 16:41:06.314434635 (769h12m6.18551768s) [upload=672981]: Upload finished. err: <nil>
Duration: 30.003942314s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
57 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXX.XX.XXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
5.000677717s
trace:
https://jaeger.dev.storj.io/trace/379538b707471e58
FAIL (2026-01-06 16:41:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000695361s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
13 out of 60 is failed (threshold=3) [..X......X..X.X.X........X.....XX......X..........XX...X.X..]
duration:
15.455499399s
trace:
https://jaeger.dev.storj.io/trace/76df23b0035fe317
FAIL (2026-01-06 16:41:07 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
g piece...
Jan  6 16:39:13.319055742 (769h10m13.190138787s) [noise=true , address=77.90.31.12:4616     , piece_id=JCWWA7PGIW23J4OKRJYW33S37FGDTIHBHTCLJSXXA4RH4SE7SWUA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.319538962 (769h10m13.190621997s) [noise=true , address=45.91.116.7:4014     , piece_id=65DKIVFKLQVVJEHZOT2P4KYDBHFD6URVGOFPFPABEGTOORENHXBA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.320135148 (769h10m13.191218193s) [noise=true , address=77.90.31.14:4668     , piece_id=QYCFWBXHW7BYLILL76W5MDFYQJKG6VZIZOYS3BF4OPDZLMJZXKVQ, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.320651351 (769h10m13.191734396s) [noise=true , address=145.79.227.11:4736   , piece_id=CTSU3X6OZJFWJSCLJBJ3EBGALWWGZL62ZJN436LKICHA4MLIQ2ZA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.321172283 (769h10m13.192255328s) [noise=true , address=64.49.13.27:4968     , piece_id=X4RAE7BUCM2R5NDMCCLH4TRGRCCXJBGUXZRE4QSNCFWNCJXLBSVA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.323456752 (769h10m13.194539787s) [noise=true , address=145.79.227.12:4773   , piece_id=JXMIF6EZKPD7EXZ646Z4XM6PG6I5P3EPTBJUWYQX4LMGAUFJT4EA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.325413879 (769h10m13.196496924s) [noise=true , address=145.79.227.33:4884   , piece_id=E5YFUCC75VTQBLZH3T5Q6VWEMRYAVOXNFKZQCPEP3ES64YCTXWPA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.327557260 (769h10m13.198640305s) [noise=true , address=145.79.227.6:4279    , piece_id=FNXBVE7QJDR67LLYLBXOCK3OQ7E6O5BIKKXPQMYOSMKIZSNK2EDQ, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.329566295 (769h10m13.20064934s) [noise=true , address=45.91.116.9:4282     , piece_id=GC72TML4SY7YTFQW4G3CVTOMGW5HWQLCWHBFQ3TKSZSUJUR252WA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.330791288 (769h10m13.201874323s) [noise=true , address=45.91.116.11:4139    , piece_id=NRCGKCNT7VWR4UUJE7HLXWT23K3BRUV2AHMDPOAVAXQF4UZKIPUA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.332349425 (769h10m13.20343246s) [noise=true , address=145.79.227.49:4371   , piece_id=YNFO36VNDUUPFQO4OFHIUEWUZWLYM2BUD7NF6VQEVWZHNCDSM6FA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.332771599 (769h10m13.203854634s) [noise=true , address=145.79.227.23:4557   , piece_id=QHDDTZ4DZAJCIU3I5HVA54JQLOHCCN73GSJAWZGC5JL6X4WV7ETA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.333853448 (769h10m13.204936494s) [noise=true , address=145.79.227.14:4843   , piece_id=GSANAVILLP6UTITEPDCBK5VHTTK7D2SBT7FQQI77H35I3SLNRVKA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:13.336038750 (769h10m13.207121785s) [noise=true , address=45.91.116.10:4107    , piece_id=I5OP5FXQBURDQG2LWEKGVZUBX7VV3CLD6Z2ZKKDK5UCX2RGFX3BA, seg_pos={0 0}, upload=672957]: Uploading piece...
Jan  6 16:39:14.793189300 (769h10m14.664272335s) [upload=672957]: Next returned nil segment
Jan  6 16:39:14.793201272 (769h10m14.664284307s) [upload=672957]: Waiting for error group managing segments...
Jan  6 16:39:15.209058789 (769h10m15.080141834s) [noise=true , address=145.79.227.13:4835   , piece_id=MMTT3F5VEI2NUZ4T2QWJGNCPIB3O4EGLYANWMCFK6MC3YLQX3TXA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.251874898 (769h10m15.122957933s) [noise=true , address=145.79.227.14:4843   , piece_id=GSANAVILLP6UTITEPDCBK5VHTTK7D2SBT7FQQI77H35I3SLNRVKA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.299718689 (769h10m15.170801724s) [noise=true , address=145.79.227.11:4736   , piece_id=CTSU3X6OZJFWJSCLJBJ3EBGALWWGZL62ZJN436LKICHA4MLIQ2ZA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.351276473 (769h10m15.222359518s) [noise=true , address=145.79.227.12:4773   , piece_id=JXMIF6EZKPD7EXZ646Z4XM6PG6I5P3EPTBJUWYQX4LMGAUFJT4EA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.423449492 (769h10m15.294532527s) [noise=true , address=64.49.13.27:4968     , piece_id=X4RAE7BUCM2R5NDMCCLH4TRGRCCXJBGUXZRE4QSNCFWNCJXLBSVA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.580777038 (769h10m15.451860083s) [noise=true , address=77.90.31.14:4668     , piece_id=QYCFWBXHW7BYLILL76W5MDFYQJKG6VZIZOYS3BF4OPDZLMJZXKVQ, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:15.798468394 (769h10m15.66955143s) [noise=true , address=45.91.116.10:4107    , piece_id=I5OP5FXQBURDQG2LWEKGVZUBX7VV3CLD6Z2ZKKDK5UCX2RGFX3BA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.083877398 (769h10m15.954960443s) [noise=true , address=45.91.116.9:4282     , piece_id=GC72TML4SY7YTFQW4G3CVTOMGW5HWQLCWHBFQ3TKSZSUJUR252WA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.143050622 (769h10m16.014133687s) [noise=true , address=145.79.227.32:4865   , piece_id=Z5ZJEDUJG376CVJM4EDK6JPRJK2U5OR6RDD4YG5X4MXH4WKSZYSA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.194307403 (769h10m16.065390438s) [noise=true , address=145.79.227.33:4884   , piece_id=E5YFUCC75VTQBLZH3T5Q6VWEMRYAVOXNFKZQCPEP3ES64YCTXWPA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.244019653 (769h10m16.115102688s) [noise=true , address=145.79.227.49:4371   , piece_id=YNFO36VNDUUPFQO4OFHIUEWUZWLYM2BUD7NF6VQEVWZHNCDSM6FA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.587195683 (769h10m16.458278718s) [noise=true , address=45.91.116.7:4014     , piece_id=65DKIVFKLQVVJEHZOT2P4KYDBHFD6URVGOFPFPABEGTOORENHXBA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.752240113 (769h10m16.623323148s) [noise=true , address=145.79.227.21:4501   , piece_id=KQIHUVS3N56R5HFH2S2I6OE7PBZUXAHRGFMPK57KX7UA5I7BLCSQ, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.752279798 (769h10m16.623362843s) [noise=true , address=77.90.31.12:4616     , piece_id=JCWWA7PGIW23J4OKRJYW33S37FGDTIHBHTCLJSXXA4RH4SE7SWUA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: <nil>
Jan  6 16:39:16.752291660 (769h10m16.623374695s) [seg_pos={0 0}, upload=672957]: Segment reached optimal threshold of 14 pieces.
Jan  6 16:39:16.752502462 (769h10m16.623585507s) [noise=true , address=145.79.227.6:4279    , piece_id=FNXBVE7QJDR67LLYLBXOCK3OQ7E6O5BIKKXPQMYOSMKIZSNK2EDQ, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uiA6tmZtmHDNaNtkhRtH31q4MJxn3C4bfrLAXmYz5mKmofHoG): protocol: write tcp 188.166.225.170:35010->145.79.227.6:4279: use of closed network connection; write tcp 188.166.225.170:35010->145.79.227.6:4279: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35010->145.79.227.6:4279: use of closed network connection
Jan  6 16:39:16.752510658 (769h10m16.623593693s) [noise=true , address=145.79.227.6:4279    , piece_id=FNXBVE7QJDR67LLYLBXOCK3OQ7E6O5BIKKXPQMYOSMKIZSNK2EDQ, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752633512 (769h10m16.623716557s) [noise=true , address=145.79.227.20:4454   , piece_id=W7E7EUDKE6BTN5I7TSTAMHETYKCXBU7RLYALUHELLOSTOAUFVB2Q, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SafkJPqKHLwMaqyZdLq3yJ1xdjqDtWHFrUwynmegUVwEC7y5J): protocol: expected piece hash; context canceled; EOF
Jan  6 16:39:16.752641587 (769h10m16.623724632s) [noise=true , address=145.79.227.20:4454   , piece_id=W7E7EUDKE6BTN5I7TSTAMHETYKCXBU7RLYALUHELLOSTOAUFVB2Q, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752764281 (769h10m16.623847316s) [noise=true , address=145.79.227.23:4557   , piece_id=QHDDTZ4DZAJCIU3I5HVA54JQLOHCCN73GSJAWZGC5JL6X4WV7ETA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sF7TCu1pL8mrtjoPULJ2VnZQHcJSMmvou2w7YgF58ZHQcnfJT): protocol: write tcp 188.166.225.170:43038->145.79.227.23:4557: use of closed network connection; write tcp 188.166.225.170:43038->145.79.227.23:4557: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43038->145.79.227.23:4557: use of closed network connection
Jan  6 16:39:16.752774249 (769h10m16.623857295s) [noise=true , address=145.79.227.23:4557   , piece_id=QHDDTZ4DZAJCIU3I5HVA54JQLOHCCN73GSJAWZGC5JL6X4WV7ETA, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752797775 (769h10m16.62388081s) [noise=true , address=45.91.116.11:4139    , piece_id=NRCGKCNT7VWR4UUJE7HLXWT23K3BRUV2AHMDPOAVAXQF4UZKIPUA, seg_pos={0 0}, upload=672957]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Tz3HEHACmHU9Gp2W2vHVuF5PXNmdsa5N8nYhwt52HCTjggqRD): protocol: read tcp 188.166.225.170:44870->45.91.116.11:4139: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan  6 16:39:16.752805439 (769h10m16.623888484s) [noise=true , address=45.91.116.11:4139    , piece_id=NRCGKCNT7VWR4UUJE7HLXWT23K3BRUV2AHMDPOAVAXQF4UZKIPUA, seg_pos={0 0}, upload=672957]: Long tail cancellation due to optimal threshold
Jan  6 16:39:16.752825017 (769h10m16.623908052s) [seg_pos={0 0}, upload=672957]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Jan  6 16:39:16.752902604 (769h10m16.623985659s) [upload=672957]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:39:16.752918193 (769h10m16.624001229s) [upload=672957]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:39:17.044336649 (769h10m16.915419684s) [upload=672957]: Done uploading segments...
Jan  6 16:39:17.044357138 (769h10m16.915440173s) [upload=672957]: Upload finished. err: <nil>
Duration: 15.455533655s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
57 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXX.XX.XXXXXXXXXXXXXX.XXXXXXXXXXXXXXXXXXXXXXXXXX]
duration:
5.000979963s
trace:
https://jaeger.dev.storj.io/trace/3f76ec5e450295f3
FAIL (2026-01-06 16:40:52 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001006292s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
3 out of 60 is failed (threshold=3) [..X......................X......X...........................]
duration:
21.160679766s
trace:
https://jaeger.dev.storj.io/trace/d5e2d8b8127d900
FAIL (2026-01-06 16:40:41 UTC)
orbiter warning: context deadline exceeded
Duration: 21.160736585s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)
antiblip:
2 out of 60 is failed (threshold=3) [....................................................XX......]
duration:
10.890766408s
trace:
https://jaeger.dev.storj.io/trace/4b3b841c440bfc62
FAIL (2026-01-06 16:40:34 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
JXEZNOOX75ENDV6EBXU4NR4HW4GTZMVA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913312874 (769h5m40.784395909s) [noise=true , address=81.171.14.70:4035    , piece_id=BRXIJD7JX7L37MWMTVPL5KSESQUQEZNRXLBLL26XDMYIJSCRUD5A, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913698277 (769h5m40.784781322s) [noise=true , address=81.171.14.70:4030    , piece_id=WV5BULPUQSAE7DUO2FWY74KF3MIHSQXYC673GNJ7XUBQXXM6KHEQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.913862180 (769h5m40.784945215s) [noise=true , address=89.149.201.147:4004  , piece_id=NMPCYNSE3YVIAVDCWRO4VP6IN4DKZ2P4YIFYI6ZDTEU75QSFC4NQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.914346662 (769h5m40.785429707s) [noise=true , address=185.167.204.205:4035 , piece_id=HPHYG44QNKJYFSXEBH6FPYLFXLO6OUIVGDYL5GXJMP3PDNFUZEQA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.914842456 (769h5m40.785925491s) [noise=true , address=81.171.14.70:4034    , piece_id=5LGDXPCHMOQPZGJAEH3ZSPF64RSTVAJRQFZBXVWP5D3Y36YFLUUQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.915307461 (769h5m40.786390506s) [noise=true , address=89.149.201.147:4019  , piece_id=LS3NXHELDI7H6VCTHO6LAGRDDG46ST4IF2XOFZDZ6EK756K5ACHQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.915793817 (769h5m40.786876863s) [noise=true , address=185.167.204.206:4067 , piece_id=GLITDKTEKYV4CFGL3TJXWJDFOBYPFX2UTNJFPUCZDJM3HOWVFS4Q, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.916279973 (769h5m40.787363018s) [noise=true , address=89.149.201.147:4014  , piece_id=KNV6OXYOC2H7LVZCW4ZWQRYRN5ZCMII3OO3FAS3YYQ22574KUCRQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.916759336 (769h5m40.787842371s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:40.920089677 (769h5m40.791172722s) [noise=true , address=185.167.204.205:4014 , piece_id=XUCDQV4VW4ZZFTSZV2YEDAO5GQFP2ASW6BJFFC735UWEVIR3RNXA, seg_pos={0 0}, upload=672894]: Uploading piece...
Jan  6 16:34:41.741185400 (769h5m41.612268445s) [upload=672894]: Next returned nil segment
Jan  6 16:34:41.741201020 (769h5m41.612284055s) [upload=672894]: Waiting for error group managing segments...
Jan  6 16:34:42.136024704 (769h5m42.00710774s) [noise=true , address=185.167.204.205:4036 , piece_id=4XKAIZPEJZAVAS7P2QILKY7XIDIIRIRWH7HTN5OVXUHAL2Y3H4SQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.137413950 (769h5m42.008496995s) [noise=true , address=185.167.204.206:4051 , piece_id=XWKJM6ZIHISLSESJ2ZQQQMZTGMK5MHH6L4ZD5RXJ3SMAVASVVVOA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.137527506 (769h5m42.008610541s) [noise=true , address=185.167.204.206:4049 , piece_id=4T665GS7XTBVEHUVT6SFIQW4F3TTBPDKZ57PKQHHHWWKC4P4JQPQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.138183485 (769h5m42.00926653s) [noise=true , address=185.167.204.206:4048 , piece_id=ZGQYUYRK5PK24UF4PYT67FHNPR4GVFDCJ2OEGRFENLB2GYR2LFVQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.139902609 (769h5m42.010985644s) [noise=true , address=185.167.204.206:4056 , piece_id=KYHDW5RLJTZCURXCFIP7HN3ZYLPPHK7FTO6IIGQ5VJJUPF4ICQXA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.140277843 (769h5m42.011360878s) [noise=true , address=185.167.204.206:4067 , piece_id=GLITDKTEKYV4CFGL3TJXWJDFOBYPFX2UTNJFPUCZDJM3HOWVFS4Q, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.142394303 (769h5m42.013477348s) [noise=true , address=185.167.204.205:4021 , piece_id=U4X3NZRT7MHG2JTNRVDRYIC72UDMHUQUCY4VFW7JFVLLVACZN7OA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.179529514 (769h5m42.050612549s) [noise=true , address=185.167.204.206:4053 , piece_id=HBFCBJCOLPJLNE4INHP4JXEZNOOX75ENDV6EBXU4NR4HW4GTZMVA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.194387988 (769h5m42.065471023s) [noise=true , address=185.167.204.205:4035 , piece_id=HPHYG44QNKJYFSXEBH6FPYLFXLO6OUIVGDYL5GXJMP3PDNFUZEQA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.197860059 (769h5m42.068943094s) [noise=true , address=185.167.204.206:4064 , piece_id=JGCXE3QUORVKLRIWEXFZUCT4XUF3DRVA35ZKQU4DQODLZBOH5I4A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.827615690 (769h5m42.698698725s) [noise=true , address=81.171.14.70:4041    , piece_id=NLGMNQJDFISSAHBX3EC6WMWJLP74FMR4MRPUIX4GCCCQRG5RRRCQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.917676689 (769h5m42.788759725s) [noise=true , address=81.171.14.70:4034    , piece_id=5LGDXPCHMOQPZGJAEH3ZSPF64RSTVAJRQFZBXVWP5D3Y36YFLUUQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.932213350 (769h5m42.803296385s) [noise=true , address=81.171.14.70:4030    , piece_id=WV5BULPUQSAE7DUO2FWY74KF3MIHSQXYC673GNJ7XUBQXXM6KHEQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:42.956051121 (769h5m42.827134196s) [noise=true , address=81.171.14.70:4031    , piece_id=OJKJHNVWTPDITZC5CTQNB7UFPIVZDPE54ZXS4LWRM3XHYAB6443A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.005259852 (769h5m42.876342887s) [noise=true , address=185.167.204.205:4019 , piece_id=I7OP7AQS2DC3C7GZWG5RVJSLDKAZD2434WMBPRYMZFVAJYJG3JHQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.052708792 (769h5m42.923791837s) [noise=true , address=81.171.14.70:4046    , piece_id=OV3PMCD6UIFNSQNTCOS3GEQC4676KFJOUH5ZX576OFOYC5EXCZPQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.639749858 (769h5m43.510832903s) [noise=true , address=89.149.201.147:4004  , piece_id=NMPCYNSE3YVIAVDCWRO4VP6IN4DKZ2P4YIFYI6ZDTEU75QSFC4NQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.719270598 (769h5m43.590353643s) [noise=true , address=89.149.201.147:4019  , piece_id=LS3NXHELDI7H6VCTHO6LAGRDDG46ST4IF2XOFZDZ6EK756K5ACHQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.892052420 (769h5m43.763135466s) [noise=true , address=81.171.14.70:4035    , piece_id=BRXIJD7JX7L37MWMTVPL5KSESQUQEZNRXLBLL26XDMYIJSCRUD5A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:43.901787498 (769h5m43.772870533s) [noise=true , address=81.171.14.70:4039    , piece_id=VFXOSS5DRL43RDJGPLSH3667HWKAF5ZNC2CF5BA7TDR4NSCG75HA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.019887208 (769h5m43.890970243s) [noise=true , address=185.167.204.206:4050 , piece_id=KSCVNGXUGFV7XRNU5MFIKLLQGZ5BNHTIXQLVHWPMITZOQUA23DGQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.510337921 (769h5m44.381420957s) [noise=true , address=185.167.204.205:4028 , piece_id=QPB4EPPEENW277S4SBIOCWWMBLIE6GKCGEM3TV3MXN6SYFMQAI7Q, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:44.692198242 (769h5m44.563281277s) [noise=true , address=89.149.201.147:4010  , piece_id=C57ORVBIZE4QUWCVG3OVG3KW2Z2GYVWUXEOYG5R53KV2WUQLR7JQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.394574949 (769h5m46.265657984s) [noise=true , address=89.149.201.147:4014  , piece_id=KNV6OXYOC2H7LVZCW4ZWQRYRN5ZCMII3OO3FAS3YYQ22574KUCRQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.543864889 (769h5m46.414947934s) [noise=true , address=185.167.204.205:4014 , piece_id=XUCDQV4VW4ZZFTSZV2YEDAO5GQFP2ASW6BJFFC735UWEVIR3RNXA, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.846238946 (769h5m46.717321982s) [noise=true , address=185.167.204.205:4006 , piece_id=W3XXEI7MG37VJHHK2CECNKIIHB7WSQABGVB5W5ZGAWSTFQ4NQMEQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: <nil>
Jan  6 16:34:46.846258273 (769h5m46.717341308s) [seg_pos={0 0}, upload=672894]: Segment reached optimal threshold of 26 pieces.
Jan  6 16:34:46.846448986 (769h5m46.717532021s) [noise=true , address=185.167.204.206:4072 , piece_id=EJ4XCLNDZETIFIKE3FK2Q4NA35KT5LOTNGE253MKUCAIGH4FZD5A, seg_pos={0 0}, upload=672894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:17uNR1rhasAqhitGStEMyeW7EmHrz7R87jr5nJB7S7CbZvy59v): protocol: expected piece hash; context canceled; EOF
Jan  6 16:34:46.846454998 (769h5m46.717538033s) [noise=true , address=185.167.204.206:4072 , piece_id=EJ4XCLNDZETIFIKE3FK2Q4NA35KT5LOTNGE253MKUCAIGH4FZD5A, seg_pos={0 0}, upload=672894]: Long tail cancellation due to optimal threshold
Jan  6 16:34:46.846579175 (769h5m46.71766221s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14cSz6XLmfQYFmiCDqybBjNJGY9tUaNxsk8MfKc1ZHxZtRirZK): protocol: expected piece hash; context canceled; EOF
Jan  6 16:34:46.846595475 (769h5m46.71767851s) [noise=true , address=89.149.201.147:4005  , piece_id=HWCYTAY5UCUFZSKP423EG3HS7KKKFZZB3PDV5WDKJTQ3PYETMVIQ, seg_pos={0 0}, upload=672894]: Long tail cancellation due to optimal threshold
Jan  6 16:34:46.846620453 (769h5m46.717703488s) [seg_pos={0 0}, upload=672894]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Jan  6 16:34:46.846729380 (769h5m46.717812415s) [upload=672894]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jan  6 16:34:46.846734600 (769h5m46.717817635s) [upload=672894]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jan  6 16:34:47.079572108 (769h5m46.950655143s) [upload=672894]: Done uploading segments...
Jan  6 16:34:47.079590854 (769h5m46.950673889s) [upload=672894]: Upload finished. err: <nil>
Duration: 10.890791306s core uplink eu-select-1 eu1-placement-15