Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.............................................X.....X........] duration:
3.923676104s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:22:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.923700481s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..............................................X.....X.......] duration:
3.918766065s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:22:18 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.91878956s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...............................................X.....X......] duration:
3.966858833s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:21:05 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.966874092s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [................................................X.....X.....] duration:
4.160616611s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:20:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.160636809s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.................................................X.....X....] duration:
6.815448641s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:19:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.815459913s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..................................................X.....X...] duration:
51.038134639s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:19:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 51.038157424s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...................................................X.....X..] duration:
5.681931073s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:18:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.681954007s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [....................................................X.....X.] duration:
4.113801526s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:17:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.113813808s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.....................................................X.....X] duration:
1m35.726315553s trace:
https://jaeger.dev.storj.io/trace/5b8e87c56e986b3
FAIL (2026-05-25 05:17:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126432973 (3h12m22.909773072s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126458632 (3h12m22.909798731s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126700243 (3h12m22.910040342s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126726463 (3h12m22.910066562s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.126981300 (3h12m22.910321399s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127020694 (3h12m22.910360794s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127267906 (3h12m22.910608005s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127374820 (3h12m22.910714919s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.127708437 (3h12m22.911048526s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128063424 (3h12m22.911403523s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128407641 (3h12m22.91174774s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128749403 (3h12m22.912089502s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.128818404 (3h12m22.912158503s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129128315 (3h12m22.912468414s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:37.129450911 (3h12m22.91279101s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Uploading piece...
May 25 05:16:38.194398916 (3h12m23.977739005s) [upload=3077]: Next returned nil segment
May 25 05:16:38.194428983 (3h12m23.977769083s) [upload=3077]: Waiting for error group managing segments...
May 25 05:16:38.484817728 (3h12m24.268157827s) [noise=true , address=212.102.52.213:4321 , piece_id=MHIIVN2M7IHFKSML7XHS2ELWPIPSPES7HRDGNFNCKVDF2X766UNQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488053753 (3h12m24.271393852s) [noise=true , address=212.102.52.215:4380 , piece_id=PLYJUHQTSJ22VIOXEPHI5GNN3Q5W5RQNNMHLGUUD5GLPRKNUDIIQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488618430 (3h12m24.271958529s) [noise=true , address=212.102.52.216:4397 , piece_id=42KTENOBPU4PDQKJQKN6QKDOQCLQHKCYYLIJTLHCRN7VGP5WYHTQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.488708272 (3h12m24.272048371s) [noise=true , address=212.102.52.213:4325 , piece_id=YPSEI55WAE3Z67UOS6PNNXQ2SLFHVADG6635PHUPMT3JVTW3D3ZA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489617216 (3h12m24.272957315s) [noise=true , address=212.102.52.216:4413 , piece_id=ZCQMNU2QDXSNG4ZGGFCGT25CINBBKZXODEN7UF3OB27SPEUGLAVQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.489835542 (3h12m24.273175641s) [noise=true , address=212.102.52.213:4328 , piece_id=D6DMQ4TDCA66Y3ZXGZYULVJQWED4D7OM37WEXIG6XGEXRPOIUR3Q, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.490529767 (3h12m24.273869866s) [noise=true , address=212.102.52.217:4425 , piece_id=DQIP5HDUFJKFOEDORCG4L2G5RNDQFNEZ373MVQ4IOAJM3VL5GTNA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491517551 (3h12m24.27485765s) [noise=true , address=212.102.52.216:4407 , piece_id=33BVJH2MFDLXAF526B5XUF33FYWP35VAVYVP3HKJWEUNPOQN77AA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.491673630 (3h12m24.275013729s) [noise=true , address=212.102.52.216:4414 , piece_id=FT6XRRH4TJVQSHDHKEG7TQC5BZDQSDQNVFAVNSZF3TSE6AUS7LJA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.493310743 (3h12m24.276650842s) [noise=true , address=212.102.52.217:4434 , piece_id=SFZSEGBFRPMXUIUI5D554VT4F22MQAXSTEA4HNSUELRP7ULN6HFQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.495868423 (3h12m24.279208532s) [noise=true , address=212.102.52.215:4384 , piece_id=FPU3OQ5MYKAAJ3I4ZUUTGXZBE4MBAYTT5NPHFXE5LNZNA5T2YSSQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.496342578 (3h12m24.279682677s) [noise=true , address=212.102.52.214:4368 , piece_id=GTUP5HZQZJYZB2GWGL5VZXEROR3DULOYMHHWGIVE56SOZ37ID5IA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.497986694 (3h12m24.281326793s) [noise=true , address=212.102.52.213:4343 , piece_id=HGPYXHMN5X4RB5BK6V2OLEITYDSOFA5XOMLG74VFT63GVSWPJ5GA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.500459082 (3h12m24.283799181s) [noise=true , address=212.102.52.215:4392 , piece_id=WKVEZ3CH2LSMEA43CKRG32GHD655ZKZVZBBYQHPKZY2OBVYQ6ZMQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.502347404 (3h12m24.285687503s) [noise=true , address=212.102.52.214:4360 , piece_id=3OX2QJNNW7FHBSK4FHW7UO5KJP6WDBMFZQY6SKXBPB3FHTVBVKGQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505565645 (3h12m24.288905744s) [noise=true , address=212.102.52.214:4359 , piece_id=UB7H7WM2OERAIGSJXKJOXGUB6JRWFEQBSG32QNE7VH53CO6KEHIA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: <nil>
May 25 05:16:38.505577588 (3h12m24.288917687s) [seg_pos={0 0}, upload=3077]: Satisfied cohort requirements after 16 pieces
May 25 05:16:38.505936382 (3h12m24.289276502s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.505943897 (3h12m24.289283996s) [noise=true , address=212.102.52.217:4430 , piece_id=FXULHT522POF3HT3OESUGHI5WPGUG5ZFKVG6CZ4TNI7BDJ7JQLEA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506011205 (3h12m24.289351304s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506025122 (3h12m24.289365221s) [noise=true , address=212.102.52.215:4390 , piece_id=WL6ARV2S23HQ7LM7SWJ56CCXTCFZ45XJ45YGWNBAXQGKRL53RUJQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506167774 (3h12m24.289507873s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gm8ZFEuFM9SdzCZ3LsdyTM95KenMv7SAiK7dpgqj6Y7uKPyGo): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506174186 (3h12m24.289514275s) [noise=true , address=212.102.52.214:4367 , piece_id=XDDIL4RMOAAXLUA32JBFZ5YVU3WQVOTRW7Z5BQPSJRKHJSUYN5HA, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506194134 (3h12m24.289534233s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG): protocol: expected piece hash; context canceled; EOF
May 25 05:16:38.506201078 (3h12m24.289541167s) [noise=true , address=212.102.52.217:4431 , piece_id=36VI5WBQJ2TZGYUUMDQ3Q3B2QQEW4O4HJPDJHZTXJPR3ECGFT6SQ, seg_pos={0 0}, upload=3077]: Long tail cancellation due to optimal threshold
May 25 05:16:38.506223059 (3h12m24.289563158s) [seg_pos={0 0}, upload=3077]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:16:38.506332418 (3h12m24.289672517s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:16:38.506337768 (3h12m24.289677867s) [upload=3077]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:17:43.815685502 (3h13m29.599025641s) [upload=3077]: Done uploading segments...
May 25 05:17:43.815779080 (3h13m29.59911919s) [upload=3077]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1m35.726338807s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
9.287670409s trace:
https://jaeger.dev.storj.io/trace/4c9b05da8836db5a
FAIL (2026-05-25 05:15:21 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 05:11:03.036306060 (3h6m48.819646159s) [noise=true , address=212.102.52.217:4418 , piece_id=XI47RHR74YLVLO6JZ6PKSENVXYD4N2B5T62FJEVNUD2N2N32UTAQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.036698047 (3h6m48.820038157s) [noise=true , address=212.102.52.214:4351 , piece_id=Z7MO4THMPMR73DZMB7QRH3RXKU5Y2XAZSUPNYJFC72X6PCNTEPGQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.036722254 (3h6m48.820062353s) [noise=true , address=212.102.52.213:4336 , piece_id=WBMKPPZQGFSOPPEETIEINL742OLYM2ZNLCD253KPN5BQXK6Y7ODA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037107218 (3h6m48.820447327s) [noise=true , address=212.102.52.217:4417 , piece_id=FGOLUOLBECWMN3EWTQU6UQR74VWK34SA2HGYZD3KBZ5ZO4KOCKSA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037148788 (3h6m48.820488887s) [noise=true , address=212.102.52.213:4325 , piece_id=32DXIIRTYDDYYMNAQ6XFOSEAYMVXRXCGEWV5ROWZR62S5OO3FIKQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037476002 (3h6m48.820816101s) [noise=true , address=212.102.52.216:4407 , piece_id=IOVPLYB3WWGQU4SR2UEF42ZQNJZI6LD4WWMG6HPZ73RPAM2KWQXQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037533151 (3h6m48.82087325s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037871216 (3h6m48.821211315s) [noise=true , address=212.102.52.217:4438 , piece_id=7EJMGO4L6GRUT4SS56CQA6IFCQX7FM7KB75WZUT7F2DW5W5REZUQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037957020 (3h6m48.821297119s) [noise=true , address=212.102.52.216:4393 , piece_id=OP5XJELNUU2VYGDO6DX4XPREO67F4B3OWK6RFJULATXZ2WQDOUUA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038368565 (3h6m48.821708664s) [noise=true , address=212.102.52.216:4396 , piece_id=IA4HDDHF4BSB72ERPGFJUW4AQWUCAHYJ4OA4RQGD2QVBWH3OHJ6Q, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038470470 (3h6m48.821810569s) [noise=true , address=212.102.52.214:4357 , piece_id=RE4OGAKZRU36X4RBDHQ4YBKBCOQRDS7GOIGRGTZC2B3XYHE4SR4A, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038835175 (3h6m48.822175275s) [noise=true , address=212.102.52.217:4434 , piece_id=7TVUKBJN2LX6RZQJZGBLE2BIJWOKFZE6VRPPWHXNVH2XQNMEXNHA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038921791 (3h6m48.82226189s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.039337133 (3h6m48.822677242s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.039348625 (3h6m48.822688724s) [noise=true , address=212.102.52.215:4387 , piece_id=Q3FKRPIRLVWMN6S4STBCHN7TPXEXF24KYOAYPFJXM45T7UF63FOA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.042750165 (3h6m48.826090274s) [noise=true , address=212.102.52.215:4382 , piece_id=AQKF6VKBAUUL2UJC6LWMS6UEF73MO5IOP57RXKNQ4R4ZXW5MCAFA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:04.101653338 (3h6m49.884993437s) [upload=2988]: Next returned nil segment
May 25 05:11:04.101664029 (3h6m49.885004128s) [upload=2988]: Waiting for error group managing segments...
May 25 05:11:04.387475144 (3h6m50.170815243s) [noise=true , address=212.102.52.213:4322 , piece_id=VJUS6O7BUOWYSTTJEBJ47P7Y6LX7BBLDE6WKSTBTXZ64UWLLNGZQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.387804663 (3h6m50.171144762s) [noise=true , address=212.102.52.213:4330 , piece_id=B53VV6X6OYHI7JEUETEPNBJJBW5I6WJ5LUDYHWSIMDVMQE54HUWA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.389752791 (3h6m50.17309291s) [noise=true , address=212.102.52.214:4352 , piece_id=HQE2MESMB3PY2VEL32SZMPKCWFUFPSZ6HWJ6ZSUFA2PQKSLTZINA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.395430253 (3h6m50.178770373s) [noise=true , address=212.102.52.215:4387 , piece_id=Q3FKRPIRLVWMN6S4STBCHN7TPXEXF24KYOAYPFJXM45T7UF63FOA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.398606574 (3h6m50.181946673s) [noise=true , address=212.102.52.217:4434 , piece_id=7TVUKBJN2LX6RZQJZGBLE2BIJWOKFZE6VRPPWHXNVH2XQNMEXNHA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.400241934 (3h6m50.183582033s) [noise=true , address=212.102.52.217:4438 , piece_id=7EJMGO4L6GRUT4SS56CQA6IFCQX7FM7KB75WZUT7F2DW5W5REZUQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.400397681 (3h6m50.18373778s) [noise=true , address=212.102.52.213:4336 , piece_id=WBMKPPZQGFSOPPEETIEINL742OLYM2ZNLCD253KPN5BQXK6Y7ODA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.401573064 (3h6m50.184913163s) [noise=true , address=212.102.52.216:4396 , piece_id=IA4HDDHF4BSB72ERPGFJUW4AQWUCAHYJ4OA4RQGD2QVBWH3OHJ6Q, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.404003982 (3h6m50.187344081s) [noise=true , address=212.102.52.214:4351 , piece_id=Z7MO4THMPMR73DZMB7QRH3RXKU5Y2XAZSUPNYJFC72X6PCNTEPGQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.404997037 (3h6m50.188337136s) [noise=true , address=212.102.52.217:4417 , piece_id=FGOLUOLBECWMN3EWTQU6UQR74VWK34SA2HGYZD3KBZ5ZO4KOCKSA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.406888195 (3h6m50.190228294s) [noise=true , address=212.102.52.215:4382 , piece_id=AQKF6VKBAUUL2UJC6LWMS6UEF73MO5IOP57RXKNQ4R4ZXW5MCAFA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.410440744 (3h6m50.193780843s) [noise=true , address=212.102.52.213:4325 , piece_id=32DXIIRTYDDYYMNAQ6XFOSEAYMVXRXCGEWV5ROWZR62S5OO3FIKQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.410865655 (3h6m50.194205754s) [noise=true , address=212.102.52.214:4357 , piece_id=RE4OGAKZRU36X4RBDHQ4YBKBCOQRDS7GOIGRGTZC2B3XYHE4SR4A, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.411404643 (3h6m50.194744743s) [noise=true , address=212.102.52.217:4418 , piece_id=XI47RHR74YLVLO6JZ6PKSENVXYD4N2B5T62FJEVNUD2N2N32UTAQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.418261076 (3h6m50.201601165s) [noise=true , address=212.102.52.216:4407 , piece_id=IOVPLYB3WWGQU4SR2UEF42ZQNJZI6LD4WWMG6HPZ73RPAM2KWQXQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.443147300 (3h6m50.226487399s) [noise=true , address=212.102.52.216:4393 , piece_id=OP5XJELNUU2VYGDO6DX4XPREO67F4B3OWK6RFJULATXZ2WQDOUUA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.443165846 (3h6m50.226505945s) [seg_pos={0 0}, upload=2988]: Satisfied cohort requirements after 16 pieces
May 25 05:11:04.443450529 (3h6m50.226790628s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v7RCaWKsXn5ummw3iuCB5kGNH2SEvA3AedxJ5zdUj4vX3U8bn): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.443477039 (3h6m50.226817138s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.443771091 (3h6m50.22711119s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pYZV8AsdAcSvA2UfkvCsWkUZyYkW79rtVmQY8hNPUdePMzB5b): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.443778865 (3h6m50.227118954s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444200219 (3h6m50.227540308s) [noise=true , address=212.102.52.215:4375 , piece_id=DKQYLOKHURKLBDHITXC2ESIA7SK52ZXC2G7GO5UF62SSUMIII4IQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gQJgXYUaeroFf9UZMy8qCBTAYVaj3PwxbPyPsefRKpX72F2P8): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.444207052 (3h6m50.227547141s) [noise=true , address=212.102.52.215:4375 , piece_id=DKQYLOKHURKLBDHITXC2ESIA7SK52ZXC2G7GO5UF62SSUMIII4IQ, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444391544 (3h6m50.227731643s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wAfNSuNAdVkfMjxBhTsxFpMrKVephQNoWmgtBu4fvUEmGrHtD): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.444397636 (3h6m50.227737735s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444414077 (3h6m50.227754176s) [seg_pos={0 0}, upload=2988]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:11:04.444638897 (3h6m50.227978996s) [upload=2988]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:11:04.444644677 (3h6m50.227984777s) [upload=2988]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:11:06.258318569 (3h6m52.041658668s) [upload=2988]: Done uploading segments...
May 25 05:11:06.258336633 (3h6m52.041676732s) [upload=2988]: Upload finished. err: <nil>
Duration: 9.287686719s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
5.177224312s trace:
https://jaeger.dev.storj.io/trace/4c9b05da8836db5a
FAIL (2026-05-25 05:14:20 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 05:11:03.036306060 (3h6m48.819646159s) [noise=true , address=212.102.52.217:4418 , piece_id=XI47RHR74YLVLO6JZ6PKSENVXYD4N2B5T62FJEVNUD2N2N32UTAQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.036698047 (3h6m48.820038157s) [noise=true , address=212.102.52.214:4351 , piece_id=Z7MO4THMPMR73DZMB7QRH3RXKU5Y2XAZSUPNYJFC72X6PCNTEPGQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.036722254 (3h6m48.820062353s) [noise=true , address=212.102.52.213:4336 , piece_id=WBMKPPZQGFSOPPEETIEINL742OLYM2ZNLCD253KPN5BQXK6Y7ODA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037107218 (3h6m48.820447327s) [noise=true , address=212.102.52.217:4417 , piece_id=FGOLUOLBECWMN3EWTQU6UQR74VWK34SA2HGYZD3KBZ5ZO4KOCKSA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037148788 (3h6m48.820488887s) [noise=true , address=212.102.52.213:4325 , piece_id=32DXIIRTYDDYYMNAQ6XFOSEAYMVXRXCGEWV5ROWZR62S5OO3FIKQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037476002 (3h6m48.820816101s) [noise=true , address=212.102.52.216:4407 , piece_id=IOVPLYB3WWGQU4SR2UEF42ZQNJZI6LD4WWMG6HPZ73RPAM2KWQXQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037533151 (3h6m48.82087325s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037871216 (3h6m48.821211315s) [noise=true , address=212.102.52.217:4438 , piece_id=7EJMGO4L6GRUT4SS56CQA6IFCQX7FM7KB75WZUT7F2DW5W5REZUQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037957020 (3h6m48.821297119s) [noise=true , address=212.102.52.216:4393 , piece_id=OP5XJELNUU2VYGDO6DX4XPREO67F4B3OWK6RFJULATXZ2WQDOUUA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038368565 (3h6m48.821708664s) [noise=true , address=212.102.52.216:4396 , piece_id=IA4HDDHF4BSB72ERPGFJUW4AQWUCAHYJ4OA4RQGD2QVBWH3OHJ6Q, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038470470 (3h6m48.821810569s) [noise=true , address=212.102.52.214:4357 , piece_id=RE4OGAKZRU36X4RBDHQ4YBKBCOQRDS7GOIGRGTZC2B3XYHE4SR4A, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038835175 (3h6m48.822175275s) [noise=true , address=212.102.52.217:4434 , piece_id=7TVUKBJN2LX6RZQJZGBLE2BIJWOKFZE6VRPPWHXNVH2XQNMEXNHA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038921791 (3h6m48.82226189s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.039337133 (3h6m48.822677242s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.039348625 (3h6m48.822688724s) [noise=true , address=212.102.52.215:4387 , piece_id=Q3FKRPIRLVWMN6S4STBCHN7TPXEXF24KYOAYPFJXM45T7UF63FOA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.042750165 (3h6m48.826090274s) [noise=true , address=212.102.52.215:4382 , piece_id=AQKF6VKBAUUL2UJC6LWMS6UEF73MO5IOP57RXKNQ4R4ZXW5MCAFA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:04.101653338 (3h6m49.884993437s) [upload=2988]: Next returned nil segment
May 25 05:11:04.101664029 (3h6m49.885004128s) [upload=2988]: Waiting for error group managing segments...
May 25 05:11:04.387475144 (3h6m50.170815243s) [noise=true , address=212.102.52.213:4322 , piece_id=VJUS6O7BUOWYSTTJEBJ47P7Y6LX7BBLDE6WKSTBTXZ64UWLLNGZQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.387804663 (3h6m50.171144762s) [noise=true , address=212.102.52.213:4330 , piece_id=B53VV6X6OYHI7JEUETEPNBJJBW5I6WJ5LUDYHWSIMDVMQE54HUWA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.389752791 (3h6m50.17309291s) [noise=true , address=212.102.52.214:4352 , piece_id=HQE2MESMB3PY2VEL32SZMPKCWFUFPSZ6HWJ6ZSUFA2PQKSLTZINA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.395430253 (3h6m50.178770373s) [noise=true , address=212.102.52.215:4387 , piece_id=Q3FKRPIRLVWMN6S4STBCHN7TPXEXF24KYOAYPFJXM45T7UF63FOA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.398606574 (3h6m50.181946673s) [noise=true , address=212.102.52.217:4434 , piece_id=7TVUKBJN2LX6RZQJZGBLE2BIJWOKFZE6VRPPWHXNVH2XQNMEXNHA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.400241934 (3h6m50.183582033s) [noise=true , address=212.102.52.217:4438 , piece_id=7EJMGO4L6GRUT4SS56CQA6IFCQX7FM7KB75WZUT7F2DW5W5REZUQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.400397681 (3h6m50.18373778s) [noise=true , address=212.102.52.213:4336 , piece_id=WBMKPPZQGFSOPPEETIEINL742OLYM2ZNLCD253KPN5BQXK6Y7ODA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.401573064 (3h6m50.184913163s) [noise=true , address=212.102.52.216:4396 , piece_id=IA4HDDHF4BSB72ERPGFJUW4AQWUCAHYJ4OA4RQGD2QVBWH3OHJ6Q, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.404003982 (3h6m50.187344081s) [noise=true , address=212.102.52.214:4351 , piece_id=Z7MO4THMPMR73DZMB7QRH3RXKU5Y2XAZSUPNYJFC72X6PCNTEPGQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.404997037 (3h6m50.188337136s) [noise=true , address=212.102.52.217:4417 , piece_id=FGOLUOLBECWMN3EWTQU6UQR74VWK34SA2HGYZD3KBZ5ZO4KOCKSA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.406888195 (3h6m50.190228294s) [noise=true , address=212.102.52.215:4382 , piece_id=AQKF6VKBAUUL2UJC6LWMS6UEF73MO5IOP57RXKNQ4R4ZXW5MCAFA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.410440744 (3h6m50.193780843s) [noise=true , address=212.102.52.213:4325 , piece_id=32DXIIRTYDDYYMNAQ6XFOSEAYMVXRXCGEWV5ROWZR62S5OO3FIKQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.410865655 (3h6m50.194205754s) [noise=true , address=212.102.52.214:4357 , piece_id=RE4OGAKZRU36X4RBDHQ4YBKBCOQRDS7GOIGRGTZC2B3XYHE4SR4A, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.411404643 (3h6m50.194744743s) [noise=true , address=212.102.52.217:4418 , piece_id=XI47RHR74YLVLO6JZ6PKSENVXYD4N2B5T62FJEVNUD2N2N32UTAQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.418261076 (3h6m50.201601165s) [noise=true , address=212.102.52.216:4407 , piece_id=IOVPLYB3WWGQU4SR2UEF42ZQNJZI6LD4WWMG6HPZ73RPAM2KWQXQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.443147300 (3h6m50.226487399s) [noise=true , address=212.102.52.216:4393 , piece_id=OP5XJELNUU2VYGDO6DX4XPREO67F4B3OWK6RFJULATXZ2WQDOUUA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.443165846 (3h6m50.226505945s) [seg_pos={0 0}, upload=2988]: Satisfied cohort requirements after 16 pieces
May 25 05:11:04.443450529 (3h6m50.226790628s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v7RCaWKsXn5ummw3iuCB5kGNH2SEvA3AedxJ5zdUj4vX3U8bn): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.443477039 (3h6m50.226817138s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.443771091 (3h6m50.22711119s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pYZV8AsdAcSvA2UfkvCsWkUZyYkW79rtVmQY8hNPUdePMzB5b): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.443778865 (3h6m50.227118954s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444200219 (3h6m50.227540308s) [noise=true , address=212.102.52.215:4375 , piece_id=DKQYLOKHURKLBDHITXC2ESIA7SK52ZXC2G7GO5UF62SSUMIII4IQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gQJgXYUaeroFf9UZMy8qCBTAYVaj3PwxbPyPsefRKpX72F2P8): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.444207052 (3h6m50.227547141s) [noise=true , address=212.102.52.215:4375 , piece_id=DKQYLOKHURKLBDHITXC2ESIA7SK52ZXC2G7GO5UF62SSUMIII4IQ, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444391544 (3h6m50.227731643s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wAfNSuNAdVkfMjxBhTsxFpMrKVephQNoWmgtBu4fvUEmGrHtD): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.444397636 (3h6m50.227737735s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444414077 (3h6m50.227754176s) [seg_pos={0 0}, upload=2988]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:11:04.444638897 (3h6m50.227978996s) [upload=2988]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:11:04.444644677 (3h6m50.227984777s) [upload=2988]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:11:06.258318569 (3h6m52.041658668s) [upload=2988]: Done uploading segments...
May 25 05:11:06.258336633 (3h6m52.041676732s) [upload=2988]: Upload finished. err: <nil>
Duration: 5.177247186s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
12.865200247s trace:
https://jaeger.dev.storj.io/trace/4c9b05da8836db5a
FAIL (2026-05-25 05:13:18 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 05:11:03.036306060 (3h6m48.819646159s) [noise=true , address=212.102.52.217:4418 , piece_id=XI47RHR74YLVLO6JZ6PKSENVXYD4N2B5T62FJEVNUD2N2N32UTAQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.036698047 (3h6m48.820038157s) [noise=true , address=212.102.52.214:4351 , piece_id=Z7MO4THMPMR73DZMB7QRH3RXKU5Y2XAZSUPNYJFC72X6PCNTEPGQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.036722254 (3h6m48.820062353s) [noise=true , address=212.102.52.213:4336 , piece_id=WBMKPPZQGFSOPPEETIEINL742OLYM2ZNLCD253KPN5BQXK6Y7ODA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037107218 (3h6m48.820447327s) [noise=true , address=212.102.52.217:4417 , piece_id=FGOLUOLBECWMN3EWTQU6UQR74VWK34SA2HGYZD3KBZ5ZO4KOCKSA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037148788 (3h6m48.820488887s) [noise=true , address=212.102.52.213:4325 , piece_id=32DXIIRTYDDYYMNAQ6XFOSEAYMVXRXCGEWV5ROWZR62S5OO3FIKQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037476002 (3h6m48.820816101s) [noise=true , address=212.102.52.216:4407 , piece_id=IOVPLYB3WWGQU4SR2UEF42ZQNJZI6LD4WWMG6HPZ73RPAM2KWQXQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037533151 (3h6m48.82087325s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037871216 (3h6m48.821211315s) [noise=true , address=212.102.52.217:4438 , piece_id=7EJMGO4L6GRUT4SS56CQA6IFCQX7FM7KB75WZUT7F2DW5W5REZUQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037957020 (3h6m48.821297119s) [noise=true , address=212.102.52.216:4393 , piece_id=OP5XJELNUU2VYGDO6DX4XPREO67F4B3OWK6RFJULATXZ2WQDOUUA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038368565 (3h6m48.821708664s) [noise=true , address=212.102.52.216:4396 , piece_id=IA4HDDHF4BSB72ERPGFJUW4AQWUCAHYJ4OA4RQGD2QVBWH3OHJ6Q, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038470470 (3h6m48.821810569s) [noise=true , address=212.102.52.214:4357 , piece_id=RE4OGAKZRU36X4RBDHQ4YBKBCOQRDS7GOIGRGTZC2B3XYHE4SR4A, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038835175 (3h6m48.822175275s) [noise=true , address=212.102.52.217:4434 , piece_id=7TVUKBJN2LX6RZQJZGBLE2BIJWOKFZE6VRPPWHXNVH2XQNMEXNHA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038921791 (3h6m48.82226189s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.039337133 (3h6m48.822677242s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.039348625 (3h6m48.822688724s) [noise=true , address=212.102.52.215:4387 , piece_id=Q3FKRPIRLVWMN6S4STBCHN7TPXEXF24KYOAYPFJXM45T7UF63FOA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.042750165 (3h6m48.826090274s) [noise=true , address=212.102.52.215:4382 , piece_id=AQKF6VKBAUUL2UJC6LWMS6UEF73MO5IOP57RXKNQ4R4ZXW5MCAFA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:04.101653338 (3h6m49.884993437s) [upload=2988]: Next returned nil segment
May 25 05:11:04.101664029 (3h6m49.885004128s) [upload=2988]: Waiting for error group managing segments...
May 25 05:11:04.387475144 (3h6m50.170815243s) [noise=true , address=212.102.52.213:4322 , piece_id=VJUS6O7BUOWYSTTJEBJ47P7Y6LX7BBLDE6WKSTBTXZ64UWLLNGZQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.387804663 (3h6m50.171144762s) [noise=true , address=212.102.52.213:4330 , piece_id=B53VV6X6OYHI7JEUETEPNBJJBW5I6WJ5LUDYHWSIMDVMQE54HUWA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.389752791 (3h6m50.17309291s) [noise=true , address=212.102.52.214:4352 , piece_id=HQE2MESMB3PY2VEL32SZMPKCWFUFPSZ6HWJ6ZSUFA2PQKSLTZINA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.395430253 (3h6m50.178770373s) [noise=true , address=212.102.52.215:4387 , piece_id=Q3FKRPIRLVWMN6S4STBCHN7TPXEXF24KYOAYPFJXM45T7UF63FOA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.398606574 (3h6m50.181946673s) [noise=true , address=212.102.52.217:4434 , piece_id=7TVUKBJN2LX6RZQJZGBLE2BIJWOKFZE6VRPPWHXNVH2XQNMEXNHA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.400241934 (3h6m50.183582033s) [noise=true , address=212.102.52.217:4438 , piece_id=7EJMGO4L6GRUT4SS56CQA6IFCQX7FM7KB75WZUT7F2DW5W5REZUQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.400397681 (3h6m50.18373778s) [noise=true , address=212.102.52.213:4336 , piece_id=WBMKPPZQGFSOPPEETIEINL742OLYM2ZNLCD253KPN5BQXK6Y7ODA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.401573064 (3h6m50.184913163s) [noise=true , address=212.102.52.216:4396 , piece_id=IA4HDDHF4BSB72ERPGFJUW4AQWUCAHYJ4OA4RQGD2QVBWH3OHJ6Q, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.404003982 (3h6m50.187344081s) [noise=true , address=212.102.52.214:4351 , piece_id=Z7MO4THMPMR73DZMB7QRH3RXKU5Y2XAZSUPNYJFC72X6PCNTEPGQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.404997037 (3h6m50.188337136s) [noise=true , address=212.102.52.217:4417 , piece_id=FGOLUOLBECWMN3EWTQU6UQR74VWK34SA2HGYZD3KBZ5ZO4KOCKSA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.406888195 (3h6m50.190228294s) [noise=true , address=212.102.52.215:4382 , piece_id=AQKF6VKBAUUL2UJC6LWMS6UEF73MO5IOP57RXKNQ4R4ZXW5MCAFA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.410440744 (3h6m50.193780843s) [noise=true , address=212.102.52.213:4325 , piece_id=32DXIIRTYDDYYMNAQ6XFOSEAYMVXRXCGEWV5ROWZR62S5OO3FIKQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.410865655 (3h6m50.194205754s) [noise=true , address=212.102.52.214:4357 , piece_id=RE4OGAKZRU36X4RBDHQ4YBKBCOQRDS7GOIGRGTZC2B3XYHE4SR4A, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.411404643 (3h6m50.194744743s) [noise=true , address=212.102.52.217:4418 , piece_id=XI47RHR74YLVLO6JZ6PKSENVXYD4N2B5T62FJEVNUD2N2N32UTAQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.418261076 (3h6m50.201601165s) [noise=true , address=212.102.52.216:4407 , piece_id=IOVPLYB3WWGQU4SR2UEF42ZQNJZI6LD4WWMG6HPZ73RPAM2KWQXQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.443147300 (3h6m50.226487399s) [noise=true , address=212.102.52.216:4393 , piece_id=OP5XJELNUU2VYGDO6DX4XPREO67F4B3OWK6RFJULATXZ2WQDOUUA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.443165846 (3h6m50.226505945s) [seg_pos={0 0}, upload=2988]: Satisfied cohort requirements after 16 pieces
May 25 05:11:04.443450529 (3h6m50.226790628s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v7RCaWKsXn5ummw3iuCB5kGNH2SEvA3AedxJ5zdUj4vX3U8bn): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.443477039 (3h6m50.226817138s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.443771091 (3h6m50.22711119s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pYZV8AsdAcSvA2UfkvCsWkUZyYkW79rtVmQY8hNPUdePMzB5b): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.443778865 (3h6m50.227118954s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444200219 (3h6m50.227540308s) [noise=true , address=212.102.52.215:4375 , piece_id=DKQYLOKHURKLBDHITXC2ESIA7SK52ZXC2G7GO5UF62SSUMIII4IQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gQJgXYUaeroFf9UZMy8qCBTAYVaj3PwxbPyPsefRKpX72F2P8): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.444207052 (3h6m50.227547141s) [noise=true , address=212.102.52.215:4375 , piece_id=DKQYLOKHURKLBDHITXC2ESIA7SK52ZXC2G7GO5UF62SSUMIII4IQ, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444391544 (3h6m50.227731643s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wAfNSuNAdVkfMjxBhTsxFpMrKVephQNoWmgtBu4fvUEmGrHtD): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.444397636 (3h6m50.227737735s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444414077 (3h6m50.227754176s) [seg_pos={0 0}, upload=2988]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:11:04.444638897 (3h6m50.227978996s) [upload=2988]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:11:04.444644677 (3h6m50.227984777s) [upload=2988]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:11:06.258318569 (3h6m52.041658668s) [upload=2988]: Done uploading segments...
May 25 05:11:06.258336633 (3h6m52.041676732s) [upload=2988]: Upload finished. err: <nil>
Duration: 12.865220586s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
4.182117415s trace:
https://jaeger.dev.storj.io/trace/4c9b05da8836db5a
FAIL (2026-05-25 05:12:06 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 05:11:03.036306060 (3h6m48.819646159s) [noise=true , address=212.102.52.217:4418 , piece_id=XI47RHR74YLVLO6JZ6PKSENVXYD4N2B5T62FJEVNUD2N2N32UTAQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.036698047 (3h6m48.820038157s) [noise=true , address=212.102.52.214:4351 , piece_id=Z7MO4THMPMR73DZMB7QRH3RXKU5Y2XAZSUPNYJFC72X6PCNTEPGQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.036722254 (3h6m48.820062353s) [noise=true , address=212.102.52.213:4336 , piece_id=WBMKPPZQGFSOPPEETIEINL742OLYM2ZNLCD253KPN5BQXK6Y7ODA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037107218 (3h6m48.820447327s) [noise=true , address=212.102.52.217:4417 , piece_id=FGOLUOLBECWMN3EWTQU6UQR74VWK34SA2HGYZD3KBZ5ZO4KOCKSA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037148788 (3h6m48.820488887s) [noise=true , address=212.102.52.213:4325 , piece_id=32DXIIRTYDDYYMNAQ6XFOSEAYMVXRXCGEWV5ROWZR62S5OO3FIKQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037476002 (3h6m48.820816101s) [noise=true , address=212.102.52.216:4407 , piece_id=IOVPLYB3WWGQU4SR2UEF42ZQNJZI6LD4WWMG6HPZ73RPAM2KWQXQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037533151 (3h6m48.82087325s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037871216 (3h6m48.821211315s) [noise=true , address=212.102.52.217:4438 , piece_id=7EJMGO4L6GRUT4SS56CQA6IFCQX7FM7KB75WZUT7F2DW5W5REZUQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037957020 (3h6m48.821297119s) [noise=true , address=212.102.52.216:4393 , piece_id=OP5XJELNUU2VYGDO6DX4XPREO67F4B3OWK6RFJULATXZ2WQDOUUA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038368565 (3h6m48.821708664s) [noise=true , address=212.102.52.216:4396 , piece_id=IA4HDDHF4BSB72ERPGFJUW4AQWUCAHYJ4OA4RQGD2QVBWH3OHJ6Q, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038470470 (3h6m48.821810569s) [noise=true , address=212.102.52.214:4357 , piece_id=RE4OGAKZRU36X4RBDHQ4YBKBCOQRDS7GOIGRGTZC2B3XYHE4SR4A, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038835175 (3h6m48.822175275s) [noise=true , address=212.102.52.217:4434 , piece_id=7TVUKBJN2LX6RZQJZGBLE2BIJWOKFZE6VRPPWHXNVH2XQNMEXNHA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038921791 (3h6m48.82226189s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.039337133 (3h6m48.822677242s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.039348625 (3h6m48.822688724s) [noise=true , address=212.102.52.215:4387 , piece_id=Q3FKRPIRLVWMN6S4STBCHN7TPXEXF24KYOAYPFJXM45T7UF63FOA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.042750165 (3h6m48.826090274s) [noise=true , address=212.102.52.215:4382 , piece_id=AQKF6VKBAUUL2UJC6LWMS6UEF73MO5IOP57RXKNQ4R4ZXW5MCAFA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:04.101653338 (3h6m49.884993437s) [upload=2988]: Next returned nil segment
May 25 05:11:04.101664029 (3h6m49.885004128s) [upload=2988]: Waiting for error group managing segments...
May 25 05:11:04.387475144 (3h6m50.170815243s) [noise=true , address=212.102.52.213:4322 , piece_id=VJUS6O7BUOWYSTTJEBJ47P7Y6LX7BBLDE6WKSTBTXZ64UWLLNGZQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.387804663 (3h6m50.171144762s) [noise=true , address=212.102.52.213:4330 , piece_id=B53VV6X6OYHI7JEUETEPNBJJBW5I6WJ5LUDYHWSIMDVMQE54HUWA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.389752791 (3h6m50.17309291s) [noise=true , address=212.102.52.214:4352 , piece_id=HQE2MESMB3PY2VEL32SZMPKCWFUFPSZ6HWJ6ZSUFA2PQKSLTZINA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.395430253 (3h6m50.178770373s) [noise=true , address=212.102.52.215:4387 , piece_id=Q3FKRPIRLVWMN6S4STBCHN7TPXEXF24KYOAYPFJXM45T7UF63FOA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.398606574 (3h6m50.181946673s) [noise=true , address=212.102.52.217:4434 , piece_id=7TVUKBJN2LX6RZQJZGBLE2BIJWOKFZE6VRPPWHXNVH2XQNMEXNHA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.400241934 (3h6m50.183582033s) [noise=true , address=212.102.52.217:4438 , piece_id=7EJMGO4L6GRUT4SS56CQA6IFCQX7FM7KB75WZUT7F2DW5W5REZUQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.400397681 (3h6m50.18373778s) [noise=true , address=212.102.52.213:4336 , piece_id=WBMKPPZQGFSOPPEETIEINL742OLYM2ZNLCD253KPN5BQXK6Y7ODA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.401573064 (3h6m50.184913163s) [noise=true , address=212.102.52.216:4396 , piece_id=IA4HDDHF4BSB72ERPGFJUW4AQWUCAHYJ4OA4RQGD2QVBWH3OHJ6Q, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.404003982 (3h6m50.187344081s) [noise=true , address=212.102.52.214:4351 , piece_id=Z7MO4THMPMR73DZMB7QRH3RXKU5Y2XAZSUPNYJFC72X6PCNTEPGQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.404997037 (3h6m50.188337136s) [noise=true , address=212.102.52.217:4417 , piece_id=FGOLUOLBECWMN3EWTQU6UQR74VWK34SA2HGYZD3KBZ5ZO4KOCKSA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.406888195 (3h6m50.190228294s) [noise=true , address=212.102.52.215:4382 , piece_id=AQKF6VKBAUUL2UJC6LWMS6UEF73MO5IOP57RXKNQ4R4ZXW5MCAFA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.410440744 (3h6m50.193780843s) [noise=true , address=212.102.52.213:4325 , piece_id=32DXIIRTYDDYYMNAQ6XFOSEAYMVXRXCGEWV5ROWZR62S5OO3FIKQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.410865655 (3h6m50.194205754s) [noise=true , address=212.102.52.214:4357 , piece_id=RE4OGAKZRU36X4RBDHQ4YBKBCOQRDS7GOIGRGTZC2B3XYHE4SR4A, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.411404643 (3h6m50.194744743s) [noise=true , address=212.102.52.217:4418 , piece_id=XI47RHR74YLVLO6JZ6PKSENVXYD4N2B5T62FJEVNUD2N2N32UTAQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.418261076 (3h6m50.201601165s) [noise=true , address=212.102.52.216:4407 , piece_id=IOVPLYB3WWGQU4SR2UEF42ZQNJZI6LD4WWMG6HPZ73RPAM2KWQXQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.443147300 (3h6m50.226487399s) [noise=true , address=212.102.52.216:4393 , piece_id=OP5XJELNUU2VYGDO6DX4XPREO67F4B3OWK6RFJULATXZ2WQDOUUA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.443165846 (3h6m50.226505945s) [seg_pos={0 0}, upload=2988]: Satisfied cohort requirements after 16 pieces
May 25 05:11:04.443450529 (3h6m50.226790628s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v7RCaWKsXn5ummw3iuCB5kGNH2SEvA3AedxJ5zdUj4vX3U8bn): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.443477039 (3h6m50.226817138s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.443771091 (3h6m50.22711119s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pYZV8AsdAcSvA2UfkvCsWkUZyYkW79rtVmQY8hNPUdePMzB5b): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.443778865 (3h6m50.227118954s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444200219 (3h6m50.227540308s) [noise=true , address=212.102.52.215:4375 , piece_id=DKQYLOKHURKLBDHITXC2ESIA7SK52ZXC2G7GO5UF62SSUMIII4IQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gQJgXYUaeroFf9UZMy8qCBTAYVaj3PwxbPyPsefRKpX72F2P8): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.444207052 (3h6m50.227547141s) [noise=true , address=212.102.52.215:4375 , piece_id=DKQYLOKHURKLBDHITXC2ESIA7SK52ZXC2G7GO5UF62SSUMIII4IQ, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444391544 (3h6m50.227731643s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wAfNSuNAdVkfMjxBhTsxFpMrKVephQNoWmgtBu4fvUEmGrHtD): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.444397636 (3h6m50.227737735s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444414077 (3h6m50.227754176s) [seg_pos={0 0}, upload=2988]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:11:04.444638897 (3h6m50.227978996s) [upload=2988]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:11:04.444644677 (3h6m50.227984777s) [upload=2988]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:11:06.258318569 (3h6m52.041658668s) [upload=2988]: Done uploading segments...
May 25 05:11:06.258336633 (3h6m52.041676732s) [upload=2988]: Upload finished. err: <nil>
Duration: 4.1821357s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
4.783874457s trace:
https://jaeger.dev.storj.io/trace/4c9b05da8836db5a
FAIL (2026-05-25 05:11:22 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 05:11:03.036306060 (3h6m48.819646159s) [noise=true , address=212.102.52.217:4418 , piece_id=XI47RHR74YLVLO6JZ6PKSENVXYD4N2B5T62FJEVNUD2N2N32UTAQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.036698047 (3h6m48.820038157s) [noise=true , address=212.102.52.214:4351 , piece_id=Z7MO4THMPMR73DZMB7QRH3RXKU5Y2XAZSUPNYJFC72X6PCNTEPGQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.036722254 (3h6m48.820062353s) [noise=true , address=212.102.52.213:4336 , piece_id=WBMKPPZQGFSOPPEETIEINL742OLYM2ZNLCD253KPN5BQXK6Y7ODA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037107218 (3h6m48.820447327s) [noise=true , address=212.102.52.217:4417 , piece_id=FGOLUOLBECWMN3EWTQU6UQR74VWK34SA2HGYZD3KBZ5ZO4KOCKSA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037148788 (3h6m48.820488887s) [noise=true , address=212.102.52.213:4325 , piece_id=32DXIIRTYDDYYMNAQ6XFOSEAYMVXRXCGEWV5ROWZR62S5OO3FIKQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037476002 (3h6m48.820816101s) [noise=true , address=212.102.52.216:4407 , piece_id=IOVPLYB3WWGQU4SR2UEF42ZQNJZI6LD4WWMG6HPZ73RPAM2KWQXQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037533151 (3h6m48.82087325s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037871216 (3h6m48.821211315s) [noise=true , address=212.102.52.217:4438 , piece_id=7EJMGO4L6GRUT4SS56CQA6IFCQX7FM7KB75WZUT7F2DW5W5REZUQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037957020 (3h6m48.821297119s) [noise=true , address=212.102.52.216:4393 , piece_id=OP5XJELNUU2VYGDO6DX4XPREO67F4B3OWK6RFJULATXZ2WQDOUUA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038368565 (3h6m48.821708664s) [noise=true , address=212.102.52.216:4396 , piece_id=IA4HDDHF4BSB72ERPGFJUW4AQWUCAHYJ4OA4RQGD2QVBWH3OHJ6Q, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038470470 (3h6m48.821810569s) [noise=true , address=212.102.52.214:4357 , piece_id=RE4OGAKZRU36X4RBDHQ4YBKBCOQRDS7GOIGRGTZC2B3XYHE4SR4A, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038835175 (3h6m48.822175275s) [noise=true , address=212.102.52.217:4434 , piece_id=7TVUKBJN2LX6RZQJZGBLE2BIJWOKFZE6VRPPWHXNVH2XQNMEXNHA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038921791 (3h6m48.82226189s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.039337133 (3h6m48.822677242s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.039348625 (3h6m48.822688724s) [noise=true , address=212.102.52.215:4387 , piece_id=Q3FKRPIRLVWMN6S4STBCHN7TPXEXF24KYOAYPFJXM45T7UF63FOA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.042750165 (3h6m48.826090274s) [noise=true , address=212.102.52.215:4382 , piece_id=AQKF6VKBAUUL2UJC6LWMS6UEF73MO5IOP57RXKNQ4R4ZXW5MCAFA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:04.101653338 (3h6m49.884993437s) [upload=2988]: Next returned nil segment
May 25 05:11:04.101664029 (3h6m49.885004128s) [upload=2988]: Waiting for error group managing segments...
May 25 05:11:04.387475144 (3h6m50.170815243s) [noise=true , address=212.102.52.213:4322 , piece_id=VJUS6O7BUOWYSTTJEBJ47P7Y6LX7BBLDE6WKSTBTXZ64UWLLNGZQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.387804663 (3h6m50.171144762s) [noise=true , address=212.102.52.213:4330 , piece_id=B53VV6X6OYHI7JEUETEPNBJJBW5I6WJ5LUDYHWSIMDVMQE54HUWA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.389752791 (3h6m50.17309291s) [noise=true , address=212.102.52.214:4352 , piece_id=HQE2MESMB3PY2VEL32SZMPKCWFUFPSZ6HWJ6ZSUFA2PQKSLTZINA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.395430253 (3h6m50.178770373s) [noise=true , address=212.102.52.215:4387 , piece_id=Q3FKRPIRLVWMN6S4STBCHN7TPXEXF24KYOAYPFJXM45T7UF63FOA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.398606574 (3h6m50.181946673s) [noise=true , address=212.102.52.217:4434 , piece_id=7TVUKBJN2LX6RZQJZGBLE2BIJWOKFZE6VRPPWHXNVH2XQNMEXNHA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.400241934 (3h6m50.183582033s) [noise=true , address=212.102.52.217:4438 , piece_id=7EJMGO4L6GRUT4SS56CQA6IFCQX7FM7KB75WZUT7F2DW5W5REZUQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.400397681 (3h6m50.18373778s) [noise=true , address=212.102.52.213:4336 , piece_id=WBMKPPZQGFSOPPEETIEINL742OLYM2ZNLCD253KPN5BQXK6Y7ODA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.401573064 (3h6m50.184913163s) [noise=true , address=212.102.52.216:4396 , piece_id=IA4HDDHF4BSB72ERPGFJUW4AQWUCAHYJ4OA4RQGD2QVBWH3OHJ6Q, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.404003982 (3h6m50.187344081s) [noise=true , address=212.102.52.214:4351 , piece_id=Z7MO4THMPMR73DZMB7QRH3RXKU5Y2XAZSUPNYJFC72X6PCNTEPGQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.404997037 (3h6m50.188337136s) [noise=true , address=212.102.52.217:4417 , piece_id=FGOLUOLBECWMN3EWTQU6UQR74VWK34SA2HGYZD3KBZ5ZO4KOCKSA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.406888195 (3h6m50.190228294s) [noise=true , address=212.102.52.215:4382 , piece_id=AQKF6VKBAUUL2UJC6LWMS6UEF73MO5IOP57RXKNQ4R4ZXW5MCAFA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.410440744 (3h6m50.193780843s) [noise=true , address=212.102.52.213:4325 , piece_id=32DXIIRTYDDYYMNAQ6XFOSEAYMVXRXCGEWV5ROWZR62S5OO3FIKQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.410865655 (3h6m50.194205754s) [noise=true , address=212.102.52.214:4357 , piece_id=RE4OGAKZRU36X4RBDHQ4YBKBCOQRDS7GOIGRGTZC2B3XYHE4SR4A, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.411404643 (3h6m50.194744743s) [noise=true , address=212.102.52.217:4418 , piece_id=XI47RHR74YLVLO6JZ6PKSENVXYD4N2B5T62FJEVNUD2N2N32UTAQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.418261076 (3h6m50.201601165s) [noise=true , address=212.102.52.216:4407 , piece_id=IOVPLYB3WWGQU4SR2UEF42ZQNJZI6LD4WWMG6HPZ73RPAM2KWQXQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.443147300 (3h6m50.226487399s) [noise=true , address=212.102.52.216:4393 , piece_id=OP5XJELNUU2VYGDO6DX4XPREO67F4B3OWK6RFJULATXZ2WQDOUUA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.443165846 (3h6m50.226505945s) [seg_pos={0 0}, upload=2988]: Satisfied cohort requirements after 16 pieces
May 25 05:11:04.443450529 (3h6m50.226790628s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v7RCaWKsXn5ummw3iuCB5kGNH2SEvA3AedxJ5zdUj4vX3U8bn): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.443477039 (3h6m50.226817138s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.443771091 (3h6m50.22711119s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pYZV8AsdAcSvA2UfkvCsWkUZyYkW79rtVmQY8hNPUdePMzB5b): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.443778865 (3h6m50.227118954s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444200219 (3h6m50.227540308s) [noise=true , address=212.102.52.215:4375 , piece_id=DKQYLOKHURKLBDHITXC2ESIA7SK52ZXC2G7GO5UF62SSUMIII4IQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gQJgXYUaeroFf9UZMy8qCBTAYVaj3PwxbPyPsefRKpX72F2P8): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.444207052 (3h6m50.227547141s) [noise=true , address=212.102.52.215:4375 , piece_id=DKQYLOKHURKLBDHITXC2ESIA7SK52ZXC2G7GO5UF62SSUMIII4IQ, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444391544 (3h6m50.227731643s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wAfNSuNAdVkfMjxBhTsxFpMrKVephQNoWmgtBu4fvUEmGrHtD): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.444397636 (3h6m50.227737735s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444414077 (3h6m50.227754176s) [seg_pos={0 0}, upload=2988]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:11:04.444638897 (3h6m50.227978996s) [upload=2988]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:11:04.444644677 (3h6m50.227984777s) [upload=2988]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:11:06.258318569 (3h6m52.041658668s) [upload=2988]: Done uploading segments...
May 25 05:11:06.258336633 (3h6m52.041676732s) [upload=2988]: Upload finished. err: <nil>
Duration: 4.783885428s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.002768536s trace:
https://jaeger.dev.storj.io/trace/4c9b05da8836db5a
FAIL (2026-05-25 05:11:17 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 05:11:03.036306060 (3h6m48.819646159s) [noise=true , address=212.102.52.217:4418 , piece_id=XI47RHR74YLVLO6JZ6PKSENVXYD4N2B5T62FJEVNUD2N2N32UTAQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.036698047 (3h6m48.820038157s) [noise=true , address=212.102.52.214:4351 , piece_id=Z7MO4THMPMR73DZMB7QRH3RXKU5Y2XAZSUPNYJFC72X6PCNTEPGQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.036722254 (3h6m48.820062353s) [noise=true , address=212.102.52.213:4336 , piece_id=WBMKPPZQGFSOPPEETIEINL742OLYM2ZNLCD253KPN5BQXK6Y7ODA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037107218 (3h6m48.820447327s) [noise=true , address=212.102.52.217:4417 , piece_id=FGOLUOLBECWMN3EWTQU6UQR74VWK34SA2HGYZD3KBZ5ZO4KOCKSA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037148788 (3h6m48.820488887s) [noise=true , address=212.102.52.213:4325 , piece_id=32DXIIRTYDDYYMNAQ6XFOSEAYMVXRXCGEWV5ROWZR62S5OO3FIKQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037476002 (3h6m48.820816101s) [noise=true , address=212.102.52.216:4407 , piece_id=IOVPLYB3WWGQU4SR2UEF42ZQNJZI6LD4WWMG6HPZ73RPAM2KWQXQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037533151 (3h6m48.82087325s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037871216 (3h6m48.821211315s) [noise=true , address=212.102.52.217:4438 , piece_id=7EJMGO4L6GRUT4SS56CQA6IFCQX7FM7KB75WZUT7F2DW5W5REZUQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.037957020 (3h6m48.821297119s) [noise=true , address=212.102.52.216:4393 , piece_id=OP5XJELNUU2VYGDO6DX4XPREO67F4B3OWK6RFJULATXZ2WQDOUUA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038368565 (3h6m48.821708664s) [noise=true , address=212.102.52.216:4396 , piece_id=IA4HDDHF4BSB72ERPGFJUW4AQWUCAHYJ4OA4RQGD2QVBWH3OHJ6Q, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038470470 (3h6m48.821810569s) [noise=true , address=212.102.52.214:4357 , piece_id=RE4OGAKZRU36X4RBDHQ4YBKBCOQRDS7GOIGRGTZC2B3XYHE4SR4A, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038835175 (3h6m48.822175275s) [noise=true , address=212.102.52.217:4434 , piece_id=7TVUKBJN2LX6RZQJZGBLE2BIJWOKFZE6VRPPWHXNVH2XQNMEXNHA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.038921791 (3h6m48.82226189s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.039337133 (3h6m48.822677242s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.039348625 (3h6m48.822688724s) [noise=true , address=212.102.52.215:4387 , piece_id=Q3FKRPIRLVWMN6S4STBCHN7TPXEXF24KYOAYPFJXM45T7UF63FOA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:03.042750165 (3h6m48.826090274s) [noise=true , address=212.102.52.215:4382 , piece_id=AQKF6VKBAUUL2UJC6LWMS6UEF73MO5IOP57RXKNQ4R4ZXW5MCAFA, seg_pos={0 0}, upload=2988]: Uploading piece...
May 25 05:11:04.101653338 (3h6m49.884993437s) [upload=2988]: Next returned nil segment
May 25 05:11:04.101664029 (3h6m49.885004128s) [upload=2988]: Waiting for error group managing segments...
May 25 05:11:04.387475144 (3h6m50.170815243s) [noise=true , address=212.102.52.213:4322 , piece_id=VJUS6O7BUOWYSTTJEBJ47P7Y6LX7BBLDE6WKSTBTXZ64UWLLNGZQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.387804663 (3h6m50.171144762s) [noise=true , address=212.102.52.213:4330 , piece_id=B53VV6X6OYHI7JEUETEPNBJJBW5I6WJ5LUDYHWSIMDVMQE54HUWA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.389752791 (3h6m50.17309291s) [noise=true , address=212.102.52.214:4352 , piece_id=HQE2MESMB3PY2VEL32SZMPKCWFUFPSZ6HWJ6ZSUFA2PQKSLTZINA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.395430253 (3h6m50.178770373s) [noise=true , address=212.102.52.215:4387 , piece_id=Q3FKRPIRLVWMN6S4STBCHN7TPXEXF24KYOAYPFJXM45T7UF63FOA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.398606574 (3h6m50.181946673s) [noise=true , address=212.102.52.217:4434 , piece_id=7TVUKBJN2LX6RZQJZGBLE2BIJWOKFZE6VRPPWHXNVH2XQNMEXNHA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.400241934 (3h6m50.183582033s) [noise=true , address=212.102.52.217:4438 , piece_id=7EJMGO4L6GRUT4SS56CQA6IFCQX7FM7KB75WZUT7F2DW5W5REZUQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.400397681 (3h6m50.18373778s) [noise=true , address=212.102.52.213:4336 , piece_id=WBMKPPZQGFSOPPEETIEINL742OLYM2ZNLCD253KPN5BQXK6Y7ODA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.401573064 (3h6m50.184913163s) [noise=true , address=212.102.52.216:4396 , piece_id=IA4HDDHF4BSB72ERPGFJUW4AQWUCAHYJ4OA4RQGD2QVBWH3OHJ6Q, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.404003982 (3h6m50.187344081s) [noise=true , address=212.102.52.214:4351 , piece_id=Z7MO4THMPMR73DZMB7QRH3RXKU5Y2XAZSUPNYJFC72X6PCNTEPGQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.404997037 (3h6m50.188337136s) [noise=true , address=212.102.52.217:4417 , piece_id=FGOLUOLBECWMN3EWTQU6UQR74VWK34SA2HGYZD3KBZ5ZO4KOCKSA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.406888195 (3h6m50.190228294s) [noise=true , address=212.102.52.215:4382 , piece_id=AQKF6VKBAUUL2UJC6LWMS6UEF73MO5IOP57RXKNQ4R4ZXW5MCAFA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.410440744 (3h6m50.193780843s) [noise=true , address=212.102.52.213:4325 , piece_id=32DXIIRTYDDYYMNAQ6XFOSEAYMVXRXCGEWV5ROWZR62S5OO3FIKQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.410865655 (3h6m50.194205754s) [noise=true , address=212.102.52.214:4357 , piece_id=RE4OGAKZRU36X4RBDHQ4YBKBCOQRDS7GOIGRGTZC2B3XYHE4SR4A, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.411404643 (3h6m50.194744743s) [noise=true , address=212.102.52.217:4418 , piece_id=XI47RHR74YLVLO6JZ6PKSENVXYD4N2B5T62FJEVNUD2N2N32UTAQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.418261076 (3h6m50.201601165s) [noise=true , address=212.102.52.216:4407 , piece_id=IOVPLYB3WWGQU4SR2UEF42ZQNJZI6LD4WWMG6HPZ73RPAM2KWQXQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.443147300 (3h6m50.226487399s) [noise=true , address=212.102.52.216:4393 , piece_id=OP5XJELNUU2VYGDO6DX4XPREO67F4B3OWK6RFJULATXZ2WQDOUUA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: <nil>
May 25 05:11:04.443165846 (3h6m50.226505945s) [seg_pos={0 0}, upload=2988]: Satisfied cohort requirements after 16 pieces
May 25 05:11:04.443450529 (3h6m50.226790628s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1v7RCaWKsXn5ummw3iuCB5kGNH2SEvA3AedxJ5zdUj4vX3U8bn): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.443477039 (3h6m50.226817138s) [noise=true , address=212.102.52.216:4406 , piece_id=FZ5LDZN26LGK3HI5IBRBOI5PJ3NNZXFBSOLETFAOC5NL3GYN3YMA, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.443771091 (3h6m50.22711119s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12pYZV8AsdAcSvA2UfkvCsWkUZyYkW79rtVmQY8hNPUdePMzB5b): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.443778865 (3h6m50.227118954s) [noise=true , address=212.102.52.214:4360 , piece_id=H4XT2C6TWAZ73RSDJVH323PSLBSGEMSBPUMQGOS4USO7ISZKOTAQ, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444200219 (3h6m50.227540308s) [noise=true , address=212.102.52.215:4375 , piece_id=DKQYLOKHURKLBDHITXC2ESIA7SK52ZXC2G7GO5UF62SSUMIII4IQ, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gQJgXYUaeroFf9UZMy8qCBTAYVaj3PwxbPyPsefRKpX72F2P8): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.444207052 (3h6m50.227547141s) [noise=true , address=212.102.52.215:4375 , piece_id=DKQYLOKHURKLBDHITXC2ESIA7SK52ZXC2G7GO5UF62SSUMIII4IQ, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444391544 (3h6m50.227731643s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12wAfNSuNAdVkfMjxBhTsxFpMrKVephQNoWmgtBu4fvUEmGrHtD): protocol: expected piece hash; context canceled; EOF
May 25 05:11:04.444397636 (3h6m50.227737735s) [noise=true , address=212.102.52.215:4370 , piece_id=QOGEIU2TYE76NJYCGNMDBXM3JLWISSO3ITVBMMKWGTGMSXDOCYNA, seg_pos={0 0}, upload=2988]: Long tail cancellation due to optimal threshold
May 25 05:11:04.444414077 (3h6m50.227754176s) [seg_pos={0 0}, upload=2988]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 05:11:04.444638897 (3h6m50.227978996s) [upload=2988]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 05:11:04.444644677 (3h6m50.227984777s) [upload=2988]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 05:11:06.258318569 (3h6m52.041658668s) [upload=2988]: Done uploading segments...
May 25 05:11:06.258336633 (3h6m52.041676732s) [upload=2988]: Upload finished. err: <nil>
Duration: 1m0.002794506s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
7.142614622s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 04:07:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.142643888s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
6.929980541s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 04:06:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.930000379s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
6.803192075s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 04:05:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.803499321s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
6.696500223s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 04:05:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.696523436s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
7.475826401s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 04:05:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.475849787s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
23.280466538s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 04:04:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 23.280486676s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
36.828693278s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 04:04:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 36.828777348s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
6.815858746s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 04:02:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.815878945s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
37.123686641s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 04:02:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 37.123706708s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
8.871719093s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 04:00:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 8.871732087s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
9.649556398s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 04:00:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 9.649582267s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
26.854935442s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 04:00:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 26.854962263s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
21.370392691s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:59:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 21.37041363s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
17.403168098s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:58:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 17.403189919s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
7.36088732s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:57:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.360900978s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
7.25502584s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:56:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.255048723s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
6.543800375s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:56:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.543844078s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
6.543367527s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:56:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.543385563s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
6.708822641s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:55:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.70884293s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
6.737602194s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:54:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.737625298s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
6.579407479s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:54:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.579433777s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
6.800033716s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:53:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.800054114s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
6.769637032s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:53:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.769660336s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
6.843479769s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:52:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.843503813s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
6.908461461s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:51:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.908484236s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
34.467793891s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:51:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 34.467807047s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [X.........................X.................................] duration:
51.814453948s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:50:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 51.814478163s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.X.........................X................................] duration:
10.504817339s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:49:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 10.504829122s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..X.........................X...............................] duration:
6.749575017s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:49:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.74959711s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [...X.........................X..............................] duration:
21.41377436s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:48:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 21.413788697s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [....X.........................X.............................] duration:
6.660886997s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:47:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.660897536s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.....X.........................X............................] duration:
37.266743098s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:47:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 37.266762386s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [......X.........................X...........................] duration:
6.672979677s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:45:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.673004885s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [.......X.........................X..........................] duration:
6.951603075s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:44:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.951632933s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [........X.........................X.........................] duration:
6.506858707s trace:
https://jaeger.dev.storj.io/trace/3006561c731233e
FAIL (2026-05-25 03:43:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.507073216s
core
uplink
us-select-1
us1-placement-20
|