Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [....................................X..........X............] duration:
4.029454433s trace:
https://jaeger.dev.storj.io/trace/50cb1c9dfcc01f98
FAIL (2026-05-26 13:02:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.029473048s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.....................................XX..........X..........] duration:
23.611711011s trace:
https://jaeger.dev.storj.io/trace/52bc4cc0477decda
FAIL (2026-05-26 13:01:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 23.611726871s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.....................................X..........X...........] duration:
3.827241361s trace:
https://jaeger.dev.storj.io/trace/50cb1c9dfcc01f98
FAIL (2026-05-26 13:01:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.827265948s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [......................................XX..........X.........] duration:
6.775892242s trace:
https://jaeger.dev.storj.io/trace/52bc4cc0477decda
FAIL (2026-05-26 13:00:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.775918131s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [..X....X...........................X........................] duration:
4.050700578s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 13:00:50 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 4.050719705s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [......................................X..........X..........] duration:
3.979490224s trace:
https://jaeger.dev.storj.io/trace/50cb1c9dfcc01f98
FAIL (2026-05-26 13:00:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.979525792s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [...X....X...........................X.......................] duration:
4.151527979s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 13:00:14 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 4.151551826s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.......................................XX..........X........] duration:
6.583638843s trace:
https://jaeger.dev.storj.io/trace/52bc4cc0477decda
FAIL (2026-05-26 13:00:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.583822364s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.......................................X..........X.........] duration:
3.811085734s trace:
https://jaeger.dev.storj.io/trace/50cb1c9dfcc01f98
FAIL (2026-05-26 12:59:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.811110872s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [....X....X...........................X......................] duration:
7.087367351s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:59:47 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 7.087395534s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [........................................XX..........X.......] duration:
6.732061126s trace:
https://jaeger.dev.storj.io/trace/52bc4cc0477decda
FAIL (2026-05-26 12:59:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.732108756s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [.....X....X...........................X.....................] duration:
3.972845967s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:59:20 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 3.972866014s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [........................................X..........X........] duration:
3.921805463s trace:
https://jaeger.dev.storj.io/trace/50cb1c9dfcc01f98
FAIL (2026-05-26 12:59:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.921829368s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [......X....X...........................X....................] duration:
3.946698914s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:58:57 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 3.946710126s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [.......X....X...........................X...................] duration:
4.139585085s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:58:42 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 4.139603552s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.........................................XX..........X......] duration:
7.062804121s trace:
https://jaeger.dev.storj.io/trace/52bc4cc0477decda
FAIL (2026-05-26 12:58:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.062828457s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.........................................X..........X.......] duration:
3.704241431s trace:
https://jaeger.dev.storj.io/trace/50cb1c9dfcc01f98
FAIL (2026-05-26 12:58:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.704253464s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [........X....X...........................X..................] duration:
4.092466222s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:57:46 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 4.092488594s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [..........................................XX..........X.....] duration:
6.703455662s trace:
https://jaeger.dev.storj.io/trace/52bc4cc0477decda
FAIL (2026-05-26 12:57:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.703480438s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [..........................................X..........X......] duration:
3.873788262s trace:
https://jaeger.dev.storj.io/trace/50cb1c9dfcc01f98
FAIL (2026-05-26 12:57:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.873831395s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [.........X....X...........................X.................] duration:
4.133224565s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:56:34 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 4.133238821s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [..........X....X...........................X................] duration:
45.555796757s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:56:30 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 45.555809251s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [...........................................XX..........X....] duration:
6.604182113s trace:
https://jaeger.dev.storj.io/trace/52bc4cc0477decda
FAIL (2026-05-26 12:56:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.604218603s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [...........................................X..........X.....] duration:
3.749863862s trace:
https://jaeger.dev.storj.io/trace/50cb1c9dfcc01f98
FAIL (2026-05-26 12:56:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.749886816s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [...........X....X...........................X...............] duration:
4.091410114s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:55:31 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 4.091433819s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [............................................XX..........X...] duration:
6.700733648s trace:
https://jaeger.dev.storj.io/trace/52bc4cc0477decda
FAIL (2026-05-26 12:55:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.700760248s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [............................................X..........X....] duration:
3.811791641s trace:
https://jaeger.dev.storj.io/trace/50cb1c9dfcc01f98
FAIL (2026-05-26 12:55:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.811820866s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.............................................XX..........X..] duration:
6.869804907s trace:
https://jaeger.dev.storj.io/trace/52bc4cc0477decda
FAIL (2026-05-26 12:55:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.869828371s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [............X....X...........................X..............] duration:
6.035368706s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:54:51 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 6.035388603s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [.............X....X...........................X.............] duration:
4.512979649s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:54:30 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 4.513004285s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [..............X....X...........................X............] duration:
3.976449484s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:54:12 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 3.976473909s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.............................................X..........X...] duration:
3.715856557s trace:
https://jaeger.dev.storj.io/trace/50cb1c9dfcc01f98
FAIL (2026-05-26 12:53:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.715894939s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [..............................................XX..........X.] duration:
6.855211554s trace:
https://jaeger.dev.storj.io/trace/52bc4cc0477decda
FAIL (2026-05-26 12:53:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.855222244s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [...............X....X...........................X...........] duration:
4.192461407s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:53:45 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 4.192483508s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [...............................................XX..........X] duration:
1m0.001419954s trace:
https://jaeger.dev.storj.io/trace/52bc4cc0477decda
FAIL (2026-05-26 12:53:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001437317s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [................X....X...........................X..........] duration:
4.702513325s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:53:20 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 4.702530728s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [..............................................X..........X..] duration:
3.903461892s trace:
https://jaeger.dev.storj.io/trace/50cb1c9dfcc01f98
FAIL (2026-05-26 12:53:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.903498222s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [.................X....X...........................X.........] duration:
4.093839841s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:52:06 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 4.093861533s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [...............................................X..........X.] duration:
3.834407779s trace:
https://jaeger.dev.storj.io/trace/50cb1c9dfcc01f98
FAIL (2026-05-26 12:52:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.8345496s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [................................................XX..........] duration:
6.50358574s trace:
https://jaeger.dev.storj.io/trace/4c2f5e8c0d911bea
FAIL (2026-05-26 12:51:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.503607492s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [................................................X..........X] duration:
5.001549744s trace:
https://jaeger.dev.storj.io/trace/50cb1c9dfcc01f98
FAIL (2026-05-26 12:51:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001569751s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [..................X....X...........................X........] duration:
9.044943566s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:51:27 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 9.0449664s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.................................................XX.........] duration:
6.648757399s trace:
https://jaeger.dev.storj.io/trace/4c2f5e8c0d911bea
FAIL (2026-05-26 12:51:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.648775733s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................................................X..........] duration:
3.711056646s trace:
https://jaeger.dev.storj.io/trace/5867dddbed7af4dc
FAIL (2026-05-26 12:50:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.711077605s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..................................................XX........] duration:
6.573901986s trace:
https://jaeger.dev.storj.io/trace/4c2f5e8c0d911bea
FAIL (2026-05-26 12:50:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.573926482s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................................................X.........] duration:
3.861909244s trace:
https://jaeger.dev.storj.io/trace/5867dddbed7af4dc
FAIL (2026-05-26 12:50:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.861926526s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [...................X....X...........................X.......] duration:
5.105675989s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:50:12 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 5.105695545s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................................................X........] duration:
3.800558087s trace:
https://jaeger.dev.storj.io/trace/5867dddbed7af4dc
FAIL (2026-05-26 12:49:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.800581472s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [....................X....X...........................X......] duration:
4.901198628s trace:
https://jaeger.dev.storj.io/trace/34a3167c93e9d452
FAIL (2026-05-26 12:49:50 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
3KJUWRAA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.173883418 (34h40m51.957223517s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174251000 (34h40m51.957591099s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174290935 (34h40m51.957631034s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174602551 (34h40m51.95794265s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174728671 (34h40m51.95806877s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.174950454 (34h40m51.958290553s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175383791 (34h40m51.95872389s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.175712058 (34h40m51.959052157s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176046917 (34h40m51.959387016s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176061394 (34h40m51.959401513s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176434937 (34h40m51.959775036s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176757021 (34h40m51.96009712s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176828588 (34h40m51.960168687s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.176863154 (34h40m51.960203253s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:06.177161373 (34h40m51.960501472s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Uploading piece...
May 26 12:45:07.250667406 (34h40m53.034007505s) [upload=33215]: Next returned nil segment
May 26 12:45:07.250679949 (34h40m53.034020048s) [upload=33215]: Waiting for error group managing segments...
May 26 12:45:07.531454635 (34h40m53.314794734s) [noise=true , address=212.102.52.217:4422 , piece_id=5DAPLRQ6JXAR7ZVIZ3TQMZUKBQ4LY4NJYEH3ZKWLSSNQ6DY7XG3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534321906 (34h40m53.317662005s) [noise=true , address=212.102.52.215:4387 , piece_id=ZDKMT2RPCY7UPUQVZFARKIW42BORNWDS5BUV47O3EX4SCXWCI2IQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.534508562 (34h40m53.317848651s) [noise=true , address=212.102.52.214:4351 , piece_id=3MMINIEVDH54UQ2VSRUQGKRWQLGEOMY6PSR5QSAOOA7X7XB7U5CQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.536279231 (34h40m53.31961934s) [noise=true , address=212.102.52.213:4329 , piece_id=IXCOW6DBJLHRVJX7XHBNSBZ5SBTFIVZUPS4VP7QDAEJ4YKWGAJ3A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.541781561 (34h40m53.32512166s) [noise=true , address=212.102.52.216:4416 , piece_id=6EBKL63GGCISIENM7D74Y5JR42ZWVUULSEDPUFN6IVK6TXF6WKIA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.565752291 (34h40m53.34909239s) [noise=true , address=212.102.52.217:4436 , piece_id=4NDBERVDRZA62QUPGSB6YHKUDOUZJMHQRFO5ZATRI7DN32WA4BOA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566131584 (34h40m53.349471683s) [noise=true , address=212.102.52.215:4380 , piece_id=3AKIGLX2AF7QPQARHGKDD35S2WKTO77OP4CFVCJCCD4I3KJUWRAA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.566578738 (34h40m53.349918837s) [noise=true , address=212.102.52.216:4411 , piece_id=2WPFP5EE74UBLHBP5NY5WTRL4AVWJUCSPY6ALGCEYL6DJM3KPOWQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.572715768 (34h40m53.356055867s) [noise=true , address=212.102.52.216:4405 , piece_id=CYRXV67TFYNXLTSPPVHFKBHFLJ7SQUXVW6E7QJJVP2IO72JWK4DA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.573001394 (34h40m53.356341493s) [noise=true , address=212.102.52.216:4393 , piece_id=XMJ5QIXQ2EQG7WEHMMSA32FMAMIBWQIJGVWCP72DLZ26NY3Y4SRQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578206936 (34h40m53.361547035s) [noise=true , address=212.102.52.217:4435 , piece_id=3MXGCBVN6N7TGMGRHE4F2SR7UFWFUMXF6F64LEBGHBAFPYB6U5UQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578665933 (34h40m53.362006042s) [noise=true , address=212.102.52.213:4328 , piece_id=DHKFJGP27IO2JG2G6TSAHS26O2XQXBGK4BO2SDP46NOLUVXB6VVQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.578932330 (34h40m53.362272429s) [noise=true , address=212.102.52.213:4336 , piece_id=JHFA274T2DK5C4UMAPKWLMBB4ZLNBYTA4BYS2ZQ2KAHYO4LZAA4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.579487169 (34h40m53.362827268s) [noise=true , address=212.102.52.214:4348 , piece_id=ZJDOAZS4B3ERZQFHZV4WNPUCLTKCBVC2GT73S7DF4M2SG52ELRZQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.581981168 (34h40m53.365321267s) [noise=true , address=212.102.52.215:4382 , piece_id=GQLFUUKEVEJJXDZVPLTWRCQBUZ5UYMDZ77CQZE5Z3AWUTCQUB6YQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584667704 (34h40m53.368007804s) [noise=true , address=212.102.52.214:4359 , piece_id=UYESE7DYK2OAUZVRTX53RCWGJIB22AQHNFFNHUJE3NAIF5MHQDMQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: <nil>
May 26 12:45:07.584689156 (34h40m53.368029255s) [seg_pos={0 0}, upload=33215]: Satisfied cohort requirements after 16 pieces
May 26 12:45:07.584881472 (34h40m53.368221581s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584891783 (34h40m53.368231882s) [noise=true , address=212.102.52.217:4424 , piece_id=XUUYFZSABTPPDUX2DJLFA5X4H2YM3ETGRVAZSOCEO5FWFQUALY4A, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.584922872 (34h40m53.368262971s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jv7h914FvJFDZb7qftHR4eNn5dnzYuJnnCg3hNRAfo6rWVcoc): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.584944583 (34h40m53.368284682s) [noise=true , address=212.102.52.215:4391 , piece_id=FKPRDILYQ4VOQ2JKHSZ5D2VPKXGPX344XQKKQ5L52MVKKSUT7EDQ, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585097405 (34h40m53.368437504s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MGexEpM3Yefmhe4mYARRJHvoGH4hNfULwt2uRgFs9fs18D9Md): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585119917 (34h40m53.368460016s) [noise=true , address=212.102.52.213:4327 , piece_id=XXFYMQI54GXXBHXFH4IJSS2RMO5PJQWYLD3ZFVVX734MXCHA2L2Q, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585152319 (34h40m53.368492428s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1286SqXMbtXfUYTRS8gDRUVFNfWEimRFetY3Z7WgZCgBwxox9R6): protocol: expected piece hash; context canceled; EOF
May 26 12:45:07.585162409 (34h40m53.368502518s) [noise=true , address=212.102.52.214:4354 , piece_id=WZIHRENPZWQEPKAVT4XOCM3N2Z4IUABP5ELNMJKJNDUQS7HU6VWA, seg_pos={0 0}, upload=33215]: Long tail cancellation due to optimal threshold
May 26 12:45:07.585248253 (34h40m53.368588362s) [seg_pos={0 0}, upload=33215]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 12:45:07.585325560 (34h40m53.368665659s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 12:45:07.585331842 (34h40m53.368671951s) [upload=33215]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 12:45:21.000413228 (34h41m6.783753337s) [upload=33215]: Done uploading segments...
May 26 12:45:21.000428477 (34h41m6.783768586s) [upload=33215]: Upload finished. err: <nil>
Duration: 4.901231092s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...................................................XX.......] duration:
10.759852474s trace:
https://jaeger.dev.storj.io/trace/4c2f5e8c0d911bea
FAIL (2026-05-26 12:49:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 10.759866772s
core
uplink
us-select-1
us1-placement-20
|