Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [X...................X......................X................] duration:
7.412550177s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:57:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.412606093s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
4.135805319s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:57:18 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 4.135833152s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.X...................X......................X...............] duration:
6.628647301s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:57:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.628667379s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
4.075753827s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:56:32 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 4.075782142s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
5.45087939s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:56:03 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 5.450906463s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [..X...................X......................X..............] duration:
6.667718003s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:55:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.667741717s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
4.18036895s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:55:32 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 4.180394888s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [X..X...................X......................X.............] duration:
6.500638474s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:55:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.500662941s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
4.12349975s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:55:13 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 4.123523594s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.X..X...................X......................X............] duration:
6.449489946s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:54:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.449511526s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
4.182782265s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:54:14 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 4.182803054s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [..X..X...................X......................X...........] duration:
6.592075832s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:54:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.592095s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [...X..X...................X......................X..........] duration:
22.499771293s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:53:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 22.499787364s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [....X..X...................X......................X.........] duration:
30.637757818s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:53:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.637777305s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
4.943444153s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:53:02 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 4.94346875s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
3.937685301s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:52:41 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 3.937700229s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.....X..X...................X......................X........] duration:
46.871482154s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:52:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 46.871505359s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
4.417979969s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:51:39 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 4.418005187s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
4.046002981s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:50:52 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 4.046017488s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [......X..X...................X......................X.......] duration:
26.658757482s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:50:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 26.65880394s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
6.771559588s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:49:54 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 6.771579878s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.......X..X...................X......................X......] duration:
20.843867653s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:49:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 20.843889533s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [........X..X...................X......................X.....] duration:
6.906088719s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:48:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.906111734s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
5.093279687s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:48:48 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 5.093305026s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.........X..X...................X......................X....] duration:
6.746342736s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:47:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.746386741s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
7.275718275s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:47:47 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 7.275738102s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [..........X..X...................X......................X...] duration:
6.697782728s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:47:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.697795701s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [...........X..X...................X......................X..] duration:
7.058802152s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:46:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.05882744s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
5.489746063s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:46:32 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 5.489773596s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [............X..X...................X......................X.] duration:
6.710625362s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:45:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.710636143s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.............X..X...................X......................X] duration:
1m0.001836481s trace:
https://jaeger.dev.storj.io/trace/68d890f77e4ed7b6
FAIL (2026-05-24 15:45:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.00184655s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
4.128825332s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:45:46 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 4.128852573s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
3.909319871s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:44:54 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 3.909349538s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [..............X..X...................X......................] duration:
43.771082207s trace:
https://jaeger.dev.storj.io/trace/673ef20d02650be4
FAIL (2026-05-24 15:44:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 43.771108568s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
4.021474411s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:44:30 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 4.021498627s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
3.942368743s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:43:16 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 3.942385836s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.002685825s trace:
https://jaeger.dev.storj.io/trace/726d967d3480d6f2
FAIL (2026-05-24 15:43:12 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
9191s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454251466 (572h54m51.975075211s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.454686537 (572h54m51.975510282s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.456338348 (572h54m51.977162083s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.457584686 (572h54m51.978408431s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.458762683 (572h54m51.979586418s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.460022868 (572h54m51.980846623s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.461193031 (572h54m51.982016786s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462221493 (572h54m51.983045238s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.462316104 (572h54m51.983139849s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463664968 (572h54m51.984488713s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.463781450 (572h54m51.984605185s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.464724820 (572h54m51.985548565s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465365813 (572h54m51.986189558s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:41.465908378 (572h54m51.986732123s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Uploading piece...
May 24 15:42:42.525990798 (572h54m53.046814543s) [upload=549502]: Next returned nil segment
May 24 15:42:42.526009154 (572h54m53.046832899s) [upload=549502]: Waiting for error group managing segments...
May 24 15:42:42.802925467 (572h54m53.323749212s) [noise=true , address=212.102.52.216:4399 , piece_id=EVDP2ZW4VIKEDMTNGJCPHC4AWGSSAGMAPFTD4U3I3VJMETZ42APA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.803691930 (572h54m53.324515665s) [noise=true , address=212.102.52.214:4367 , piece_id=2RSEE3YTK5NH5EPF6EHWR3ZY2TC6HLO3SVTUPFAHMIEN5UK6QQIQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.813999647 (572h54m53.334823392s) [noise=true , address=212.102.52.215:4386 , piece_id=JG56HLTL65E2YZ4VRBITMPVFTATV6UGBL7B7M5BQMKX6NYIBBQ6Q, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.817790400 (572h54m53.338614135s) [noise=true , address=212.102.52.214:4353 , piece_id=TB7I6F4AEOSY2CFX7VYL7CCDBPSD64XEUDRTM2MQWKK7EGH4YLAQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.819933029 (572h54m53.340756784s) [noise=true , address=212.102.52.215:4390 , piece_id=J5PUSE6Z7VIU4IH7M4JVEOMU6XZSPJA2ZWJWVBC7X5RHHLMIOYLA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.820504679 (572h54m53.341328425s) [noise=true , address=212.102.52.217:4433 , piece_id=PLGAOW45ISCAK4TNFQQQLENDC44JGJC7IT77C3UVVHXIOUS45JKA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821178916 (572h54m53.342002661s) [noise=true , address=212.102.52.213:4322 , piece_id=I3OKH57Z7EYKBDQAWTPQIDABLOGS53GCWIBEH637PYWQH34UCNQQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.821759163 (572h54m53.342582908s) [noise=true , address=212.102.52.216:4405 , piece_id=L6TQU5NZLIJ3UTWUICFU7OIDW5S7VJZUL3I53KBLK4BLC25MTE3A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826675233 (572h54m53.347498978s) [noise=true , address=212.102.52.213:4341 , piece_id=NU6LB3MDV2YIVYKGQ2UU6GXCWATGDYJJRYTDVP4V7D7YR54NH2ZQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.826801724 (572h54m53.347625469s) [noise=true , address=212.102.52.215:4382 , piece_id=F5VUJGG7CALDMLFBHZTUPF53F6FMZYV3MJPD52LF5RJRN3CEAZAA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.827620657 (572h54m53.348444402s) [noise=true , address=212.102.52.216:4407 , piece_id=XX7LX6VOWL4NLSL5ZPWAWDX45DMZJBUOEUN2DCPIE3UKWWFHHHZA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.831874443 (572h54m53.352698188s) [noise=true , address=212.102.52.217:4420 , piece_id=OJIL5LF6JE2CMRTDCZ343FUN6MJK33J5GTFTBSANXULNYVOZQPJA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.833605907 (572h54m53.354429653s) [noise=true , address=212.102.52.213:4339 , piece_id=ZGMRQI5L5GDJDESBPRAXXIX3BZRJODICDBXWPFKUFCMPO35WE5SQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.845785837 (572h54m53.366609582s) [noise=true , address=212.102.52.216:4402 , piece_id=6SK4UCNK354UGDG3CFUF5ITZKNBT46EN226R2JXS54QM52E53GYA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.865130318 (572h54m53.385954063s) [noise=true , address=212.102.52.213:4324 , piece_id=WN74PDV6UE5WK7XN76QRJJCWU5X5JPY5JEZ4EPBMTBX2M2QGYVGQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867430607 (572h54m53.388254352s) [noise=true , address=212.102.52.215:4370 , piece_id=T23U7X4SOVCPDK2WIL6H43S7DK7AQUZVVDIZWVP27VTGM7JY3PTA, seg_pos={0 0}, upload=549502]: Done uploading piece. err: <nil>
May 24 15:42:42.867449272 (572h54m53.388273017s) [seg_pos={0 0}, upload=549502]: Satisfied cohort requirements after 16 pieces
May 24 15:42:42.867622703 (572h54m53.388446448s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867628624 (572h54m53.388452369s) [noise=true , address=212.102.52.214:4358 , piece_id=445S3DDT2EVEE7PZC5FDSLGSCO6CAMB476XPHFDCFA5JU36WAU4A, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867734937 (572h54m53.388558682s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867739356 (572h54m53.388563101s) [noise=true , address=212.102.52.217:4424 , piece_id=XNLWRUBO5RBDYMCLGOIDSBLBDIVSJC7MV2AIQXRJALNQXVK4QSPQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867772449 (572h54m53.388596204s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127vua8xA3HHNDurSeqQEtNeQ7R4WJz2kTegBasccKVGrshF1dY): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867787528 (572h54m53.388611283s) [noise=true , address=212.102.52.214:4347 , piece_id=ZVAS5TWU6T4YAWAKHI3FKOWVK2MGCD624HN552YTKAVSWUIMYZNQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867834928 (572h54m53.388658673s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 24 15:42:42.867840127 (572h54m53.388663873s) [noise=true , address=212.102.52.217:4431 , piece_id=JQZHD7DXPDCP6X6JBPZCDFDOYFEEQE2QKRUW4QU46PM6XJGXCBXQ, seg_pos={0 0}, upload=549502]: Long tail cancellation due to optimal threshold
May 24 15:42:42.867872219 (572h54m53.388695974s) [seg_pos={0 0}, upload=549502]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 15:42:42.867948244 (572h54m53.388771979s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 15:42:42.867957662 (572h54m53.388781407s) [upload=549502]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 15:42:54.203307397 (572h55m4.724131142s) [upload=549502]: Done uploading segments...
May 24 15:42:54.203322785 (572h55m4.72414653s) [upload=549502]: Upload finished. err: <nil>
Duration: 1m0.002706505s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [...............X..X...................X.....................] duration:
6.850686675s trace:
https://jaeger.dev.storj.io/trace/673ef20d02650be4
FAIL (2026-05-24 15:43:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.85071035s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [................X..X...................X....................] duration:
6.622562425s trace:
https://jaeger.dev.storj.io/trace/673ef20d02650be4
FAIL (2026-05-24 15:42:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.622586421s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.................X..X...................X...................] duration:
6.778232764s trace:
https://jaeger.dev.storj.io/trace/673ef20d02650be4
FAIL (2026-05-24 15:41:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.778281767s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [..................X..X...................X..................] duration:
6.666000489s trace:
https://jaeger.dev.storj.io/trace/673ef20d02650be4
FAIL (2026-05-24 15:40:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.666034394s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [...................X..X...................X.................] duration:
6.762137485s trace:
https://jaeger.dev.storj.io/trace/673ef20d02650be4
FAIL (2026-05-24 15:39:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.762156371s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [....................X..X...................X................] duration:
6.637657602s trace:
https://jaeger.dev.storj.io/trace/673ef20d02650be4
FAIL (2026-05-24 15:38:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.63769285s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.....................X..X...................X...............] duration:
12.556414793s trace:
https://jaeger.dev.storj.io/trace/673ef20d02650be4
FAIL (2026-05-24 15:37:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 12.556433719s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [......................X..X...................X..............] duration:
13.495484357s trace:
https://jaeger.dev.storj.io/trace/673ef20d02650be4
FAIL (2026-05-24 15:36:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 13.495517752s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.......................X..X...................X.............] duration:
6.799129444s trace:
https://jaeger.dev.storj.io/trace/673ef20d02650be4
FAIL (2026-05-24 15:35:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.799141899s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [........................X..X...................X............] duration:
51.039506251s trace:
https://jaeger.dev.storj.io/trace/673ef20d02650be4
FAIL (2026-05-24 15:35:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 51.0395327s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.........................X..X...................X...........] duration:
6.680576405s trace:
https://jaeger.dev.storj.io/trace/673ef20d02650be4
FAIL (2026-05-24 15:33:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.680601013s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [..........................X..X...................X..........] duration:
6.63536337s trace:
https://jaeger.dev.storj.io/trace/673ef20d02650be4
FAIL (2026-05-24 15:33:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.635386565s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [...........................X..X...................X.........] duration:
34.116964974s trace:
https://jaeger.dev.storj.io/trace/673ef20d02650be4
FAIL (2026-05-24 15:32:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 34.116992166s
core
uplink
us-select-1
us1-placement-20
|