Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [...............................XXXXX....X.............X.X...] duration: 3.845420195s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 13:52:19 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.845444291s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................X......X....................] duration: 6.432956527s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:51:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.432983428s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X......X...................] duration: 17.856815693s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:51:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 17.856849708s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [................................XXXXX....X.............X.X..] duration: 3.959987233s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 13:51:05 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.96001223s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [...............X..................XX............X...........] duration: 7.616590988s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 13:50:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.616613331s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................X......X..................] duration: 6.342240855s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:50:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.342266815s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [.................................XXXXX....X.............X.X.] duration: 3.712634697s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 13:50:02 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.712659055s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [................X..................XX............X..........] duration: 6.949790414s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 13:49:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.949816945s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [..................................XXXXX....X.............X.X] duration: 5.000433766s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 13:49:22 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 5.000452792s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................X......X.................] duration: 6.512481024s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:49:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.5125053s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [...................................XXXXX....X.............X.] duration: 4.017726507s trace: https://jaeger.dev.storj.io/trace/69d7efdd0648f728 FAIL (2026-05-14 13:49:02 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 4.017748248s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.................X..................XX............X.........] duration: 7.068760888s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 13:49:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.068772852s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X......X................] duration: 6.854026992s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:48:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.854050967s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X......X...............] duration: 7.202318371s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:48:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.202354881s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [..................X..................XX............X........] duration: 6.772083037s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 13:48:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.772112563s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [....................................XXXXX....X.............X] duration: 5.000406383s trace: https://jaeger.dev.storj.io/trace/69d7efdd0648f728 FAIL (2026-05-14 13:47:59 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 5.000429187s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X......X..............] duration: 7.312159227s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:47:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.312194133s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [.....................................XXXXX....X.............] duration: 4.091065028s trace: https://jaeger.dev.storj.io/trace/2c53344337b8a6b6 FAIL (2026-05-14 13:47:14 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.091089355s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [...................X..................XX............X.......] duration: 7.049667001s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 13:46:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.04969285s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [......................................XXXXX....X............] duration: 4.022211789s trace: https://jaeger.dev.storj.io/trace/2c53344337b8a6b6 FAIL (2026-05-14 13:46:36 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.022236116s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [....................X..................XX............X......] duration: 6.890454236s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 13:46:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.890473703s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X......X.............] duration: 9.109122666s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:46:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 9.109138516s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.....................X..................XX............X.....] duration: 6.82996404s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 13:45:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.829986133s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [.......................................XXXXX....X...........] duration: 3.806071504s trace: https://jaeger.dev.storj.io/trace/2c53344337b8a6b6 FAIL (2026-05-14 13:45:31 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.806132701s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X......X............] duration: 6.956219997s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:45:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.95624311s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X......X...........] duration: 6.346403469s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:45:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.346431462s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X......X..........] duration: 12.918651481s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:44:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 12.918677151s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [........................................XXXXX....X..........] duration: 4.830158467s trace: https://jaeger.dev.storj.io/trace/2c53344337b8a6b6 FAIL (2026-05-14 13:44:37 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.830181032s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [......................X..................XX............X....] duration: 9.718819898s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 13:44:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 9.718844776s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X......X.........] duration: 6.751037414s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:43:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.751061029s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.......................X..................XX............X...] duration: 8.299992266s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 13:43:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.300007916s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [.........................................XXXXX....X.........] duration: 3.817809756s trace: https://jaeger.dev.storj.io/trace/2c53344337b8a6b6 FAIL (2026-05-14 13:43:35 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.817839031s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................X......X........] duration: 5.916157967s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:43:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.916184538s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [..........................................XXXXX....X........] duration: 3.955464811s trace: https://jaeger.dev.storj.io/trace/2c53344337b8a6b6 FAIL (2026-05-14 13:43:00 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.955488547s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [...........................................XXXXX....X.......] duration: 4.318947811s trace: https://jaeger.dev.storj.io/trace/2c53344337b8a6b6 FAIL (2026-05-14 13:42:32 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.318970043s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [........................X..................XX............X..] duration: 6.791048729s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 13:42:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.791069179s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.........................X..................XX............X.] duration: 8.09513052s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 13:42:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 8.095145098s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [..........................X..................XX............X] duration: 1m0.000729314s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 13:42:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000742099s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X......X.......] duration: 5.950483303s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:42:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.950504442s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X......X......] duration: 6.099533124s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:41:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.099559295s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [............................................XXXXX....X......] duration: 3.859359524s trace: https://jaeger.dev.storj.io/trace/2c53344337b8a6b6 FAIL (2026-05-14 13:41:46 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.859374474s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [.............................................XXXXX....X.....] duration: 4.196748034s trace: https://jaeger.dev.storj.io/trace/2c53344337b8a6b6 FAIL (2026-05-14 13:41:21 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.196770738s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X......X.....] duration: 7.852613892s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:40:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.852630996s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...........................X..................XX............] duration: 7.405466418s trace: https://jaeger.dev.storj.io/trace/4edc8e7010a29a3f FAIL (2026-05-14 13:40:36 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.405499141s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [..............................................XXXXX....X....] duration: 3.823194822s trace: https://jaeger.dev.storj.io/trace/2c53344337b8a6b6 FAIL (2026-05-14 13:40:25 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.82322021s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................X......X....] duration: 7.70780442s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:40:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.70783622s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [............................X..................XX...........] duration: 6.6898339s trace: https://jaeger.dev.storj.io/trace/4edc8e7010a29a3f FAIL (2026-05-14 13:39:25 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.689857704s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 6 out of 60 is failed (threshold=6) [...............................................XXXXX....X...] duration: 4.236100802s trace: https://jaeger.dev.storj.io/trace/2c53344337b8a6b6 FAIL (2026-05-14 13:39:12 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.236116502s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X......X...] duration: 5.991982866s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 13:39:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.992008545s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.............................X..................XX..........] duration: 6.608562511s trace: https://jaeger.dev.storj.io/trace/4edc8e7010a29a3f FAIL (2026-05-14 13:38:55 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.608585655s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)