Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 842081 reports (Page 1879 of 16842)
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
8 out of 60 is failed (threshold=6) [.....XXXXX....X.............X.X.............................]
duration:
3.726036745s
trace:
https://jaeger.dev.storj.io/trace/273a3254470045aa
FAIL (2026-05-14 14:10:58 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.72606043s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X]
duration:
30.000583807s
target:
https://version.storj.io/
trace:
https://jaeger.dev.storj.io/trace/d4412028e1c1a58
FAIL (2026-05-14 14:10:55 UTC)
orbiter warning: Get "https://version.storj.io/": dial tcp 35.209.24.65:443: i/o timeout
Duration: 30.000598595s core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
8 out of 60 is failed (threshold=6) [......XXXXX....X.............X.X............................]
duration:
3.880144471s
trace:
https://jaeger.dev.storj.io/trace/273a3254470045aa
FAIL (2026-05-14 14:10:26 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.880189477s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
3 out of 60 is failed (threshold=3) [..........XX............X...................................]
duration:
6.699175003s
trace:
https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43
FAIL (2026-05-14 14:10:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.699261428s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
3 out of 60 is failed (threshold=3) [...........XX............X..................................]
duration:
14.076899382s
trace:
https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43
FAIL (2026-05-14 14:10:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 14.076917786s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [......X......X..............................................]
duration:
6.382814925s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 14:10:02 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 6.382841786s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
8 out of 60 is failed (threshold=6) [.......XXXXX....X.............X.X...........................]
duration:
3.942361524s
trace:
https://jaeger.dev.storj.io/trace/273a3254470045aa
FAIL (2026-05-14 14:09:20 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.942387535s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
3 out of 60 is failed (threshold=3) [............XX............X.................................]
duration:
6.591191092s
trace:
https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43
FAIL (2026-05-14 14:09:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.591212373s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [.......X......X.............................................]
duration:
5.957648993s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 14:09:01 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 5.957670505s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
8 out of 60 is failed (threshold=6) [........XXXXX....X.............X.X..........................]
duration:
3.995994879s
trace:
https://jaeger.dev.storj.io/trace/273a3254470045aa
FAIL (2026-05-14 14:08:58 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.996098646s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [........X......X............................................]
duration:
6.965008805s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 14:08:38 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 6.965035625s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
3 out of 60 is failed (threshold=3) [.............XX............X................................]
duration:
6.789566672s
trace:
https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43
FAIL (2026-05-14 14:08:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.789588233s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
8 out of 60 is failed (threshold=6) [.........XXXXX....X.............X.X.........................]
duration:
3.957543299s
trace:
https://jaeger.dev.storj.io/trace/273a3254470045aa
FAIL (2026-05-14 14:08:06 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.957567274s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
3 out of 60 is failed (threshold=3) [..............XX............X...............................]
duration:
6.704932086s
trace:
https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43
FAIL (2026-05-14 14:08:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.704997251s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [.........X......X...........................................]
duration:
6.493910775s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 14:07:51 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 6.493938548s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
8 out of 60 is failed (threshold=6) [..........XXXXX....X.............X.X........................]
duration:
3.880852814s
trace:
https://jaeger.dev.storj.io/trace/273a3254470045aa
FAIL (2026-05-14 14:07:17 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.880876809s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [..........X......X..........................................]
duration:
6.371416992s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 14:07:14 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 6.371433323s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
3 out of 60 is failed (threshold=3) [...............XX............X..............................]
duration:
6.667443721s
trace:
https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43
FAIL (2026-05-14 14:07:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.667471665s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [...........X......X.........................................]
duration:
6.656577775s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 14:06:33 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 6.656602351s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
3 out of 60 is failed (threshold=3) [................XX............X.............................]
duration:
6.737897173s
trace:
https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43
FAIL (2026-05-14 14:06:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.73792261s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
8 out of 60 is failed (threshold=6) [...........XXXXX....X.............X.X.......................]
duration:
3.766787703s
trace:
https://jaeger.dev.storj.io/trace/273a3254470045aa
FAIL (2026-05-14 14:06:22 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.766809856s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [............X......X........................................]
duration:
7.101853171s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 14:06:12 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 7.101871485s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
3 out of 60 is failed (threshold=3) [.................XX............X............................]
duration:
13.330316385s
trace:
https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43
FAIL (2026-05-14 14:05:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 13.330330892s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [.............X......X.......................................]
duration:
6.3967895s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 14:05:09 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 6.396818416s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
8 out of 60 is failed (threshold=6) [............XXXXX....X.............X.X......................]
duration:
3.755285795s
trace:
https://jaeger.dev.storj.io/trace/273a3254470045aa
FAIL (2026-05-14 14:05:07 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.755308618s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
3 out of 60 is failed (threshold=3) [..................XX............X...........................]
duration:
6.644022899s
trace:
https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43
FAIL (2026-05-14 14:05:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.644044622s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
8 out of 60 is failed (threshold=6) [.............XXXXX....X.............X.X.....................]
duration:
3.894267924s
trace:
https://jaeger.dev.storj.io/trace/273a3254470045aa
FAIL (2026-05-14 14:04:32 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.894285678s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [..............X......X......................................]
duration:
8.478715592s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 14:03:58 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 8.478743926s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
4 out of 60 is failed (threshold=3) [X..................XX............X..........................]
duration:
6.756422346s
trace:
https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43
FAIL (2026-05-14 14:03:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.756445069s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [...............X......X.....................................]
duration:
6.315275759s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 14:03:40 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 6.315300166s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
8 out of 60 is failed (threshold=6) [..............XXXXX....X.............X.X....................]
duration:
3.850399199s
trace:
https://jaeger.dev.storj.io/trace/273a3254470045aa
FAIL (2026-05-14 14:03:35 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.850415941s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [................X......X....................................]
duration:
7.13257559s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 14:03:07 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 7.132615395s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
4 out of 60 is failed (threshold=3) [.X..................XX............X.........................]
duration:
6.897918462s
trace:
https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43
FAIL (2026-05-14 14:03:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.897997522s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
8 out of 60 is failed (threshold=6) [...............XXXXX....X.............X.X...................]
duration:
3.733275215s
trace:
https://jaeger.dev.storj.io/trace/273a3254470045aa
FAIL (2026-05-14 14:02:48 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.733313239s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [.................X......X...................................]
duration:
6.623031792s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 14:02:47 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 6.623056268s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [..................X......X..................................]
duration:
8.069063864s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 14:02:24 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 8.069088981s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
4 out of 60 is failed (threshold=3) [..X..................XX............X........................]
duration:
6.653386656s
trace:
https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43
FAIL (2026-05-14 14:02:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.653410412s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
8 out of 60 is failed (threshold=6) [................XXXXX....X.............X.X..................]
duration:
4.00625376s
trace:
https://jaeger.dev.storj.io/trace/273a3254470045aa
FAIL (2026-05-14 14:01:42 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 4.006279629s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
4 out of 60 is failed (threshold=3) [...X..................XX............X.......................]
duration:
6.813941968s
trace:
https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43
FAIL (2026-05-14 14:01:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.813965652s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [...................X......X.................................]
duration:
6.26961516s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 14:01:13 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 6.26964111s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
4 out of 60 is failed (threshold=3) [....X..................XX............X......................]
duration:
6.899970259s
trace:
https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43
FAIL (2026-05-14 14:00:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.899996029s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
8 out of 60 is failed (threshold=6) [.................XXXXX....X.............X.X.................]
duration:
3.906007085s
trace:
https://jaeger.dev.storj.io/trace/273a3254470045aa
FAIL (2026-05-14 14:00:40 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 3.90603104s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [....................X......X................................]
duration:
6.861106773s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 14:00:35 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 6.861125649s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [.....................X......X...............................]
duration:
7.58444892s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 14:00:19 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 7.584475571s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
8 out of 60 is failed (threshold=6) [..................XXXXX....X.............X.X................]
duration:
4.369674743s
trace:
https://jaeger.dev.storj.io/trace/273a3254470045aa
FAIL (2026-05-14 14:00:17 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 4.369732082s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
4 out of 60 is failed (threshold=3) [.....X..................XX............X.....................]
duration:
6.911937399s
trace:
https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43
FAIL (2026-05-14 13:59:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.911956314s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
8 out of 60 is failed (threshold=6) [...................XXXXX....X.............X.X...............]
duration:
4.181925717s
trace:
https://jaeger.dev.storj.io/trace/273a3254470045aa
FAIL (2026-05-14 13:59:38 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 4.18194832s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
8 out of 60 is failed (threshold=6) [....................XXXXX....X.............X.X..............]
duration:
4.086381318s
trace:
https://jaeger.dev.storj.io/trace/273a3254470045aa
FAIL (2026-05-14 13:59:20 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded
Duration: 4.086408341s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
2 out of 60 is failed (threshold=3) [......................X......X..............................]
duration:
6.324643598s
trace:
https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8
FAIL (2026-05-14 13:59:13 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 10)
======LOGS======
May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload
May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload
May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments...
May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false
May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment...
May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment.
May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece...
May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment
May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments...
May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716    , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762    , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979   , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542     , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618     , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486  , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812    , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663    , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517  , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816   , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil>
May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015  , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070  , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988  , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310  , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout
May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108  , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060  , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded
May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil>
May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments...
May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10)
Duration: 6.324663795s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
4 out of 60 is failed (threshold=3) [......X..................XX............X....................]
duration:
7.179064608s
trace:
https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43
FAIL (2026-05-14 13:58:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.179085859s core uplink us-select-1 us1-placement-20