Orbiter History

Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 842210 reports (Page 1881 of 16845)

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [XXXX....X.............X.X...................................] duration: 3.90424498s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 14:16:52 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.904269457s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.747670917s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 14:16:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.747694922s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 640.476352ms target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/d4412028e1c1a58 FAIL (2026-05-14 14:16:31 UTC) orbiter warning: Get "https://version.storj.io/": dial tcp 35.209.24.65:443: i/o timeout Duration: 640.502361ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...XX............X..........................................] duration: 6.812533367s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 14:16:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.812559627s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 635.85816ms target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/d4412028e1c1a58 FAIL (2026-05-14 14:15:46 UTC) orbiter warning: Get "https://version.storj.io/": dial tcp 35.209.24.65:443: i/o timeout Duration: 635.884521ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [....XX............X.........................................] duration: 6.614007063s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 14:15:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.614023305s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [XXXXX....X.............X.X..................................] duration: 3.917849298s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 14:15:38 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.917898311s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.636742586s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 14:15:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.636766001s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.....XX............X........................................] duration: 10.969715666s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 14:15:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.969739642s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 638.321933ms target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/d4412028e1c1a58 FAIL (2026-05-14 14:15:13 UTC) orbiter warning: Get "https://version.storj.io/": dial tcp 35.209.24.65:443: i/o timeout Duration: 638.349397ms core website
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 635.366915ms target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/d4412028e1c1a58 FAIL (2026-05-14 14:14:41 UTC) orbiter warning: Get "https://version.storj.io/": dial tcp 35.209.24.65:443: i/o timeout Duration: 635.390301ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [X......X....................................................] duration: 7.319813385s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 14:14:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.319832692s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [.XXXXX....X.............X.X.................................] duration: 3.858675858s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 14:14:29 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.858701056s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [......XX............X.......................................] duration: 6.70470778s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 14:14:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.704726196s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X......X...................................................] duration: 6.849886574s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 14:13:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.849909567s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [..XXXXX....X.............X.X................................] duration: 3.935841208s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 14:13:47 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.935873159s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 651.246395ms target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/d4412028e1c1a58 FAIL (2026-05-14 14:13:29 UTC) orbiter warning: Get "https://version.storj.io/": dial tcp 35.209.24.65:443: i/o timeout Duration: 651.269681ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X......X..................................................] duration: 7.181179293s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 14:13:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.181197608s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.......XX............X......................................] duration: 6.880759224s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 14:13:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.880862279s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 649.884166ms target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/d4412028e1c1a58 FAIL (2026-05-14 14:12:58 UTC) orbiter warning: Get "https://version.storj.io/": dial tcp 35.209.24.65:443: i/o timeout Duration: 649.90749ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [...XXXXX....X.............X.X...............................] duration: 3.693622003s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 14:12:42 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.693636571s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [........XX............X.....................................] duration: 6.685536306s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 14:12:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.685598986s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [....XXXXX....X.............X.X..............................] duration: 3.840057019s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 14:12:12 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.840197889s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X......X.................................................] duration: 6.55880981s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 14:12:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.55882574s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 649.17288ms target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/d4412028e1c1a58 FAIL (2026-05-14 14:11:59 UTC) orbiter warning: Get "https://version.storj.io/": dial tcp 35.209.24.65:443: i/o timeout Duration: 649.197476ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X......X................................................] duration: 7.591082772s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 14:11:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 7.591120875s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.........XX............X....................................] duration: 6.788279266s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 14:11:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.788303402s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 16.564074151s target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/d4412028e1c1a58 FAIL (2026-05-14 14:11:11 UTC) orbiter warning: Get "https://version.storj.io/": dial tcp 35.209.24.65:443: i/o timeout Duration: 16.564088829s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X......X...............................................] duration: 27.90602688s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 14:11:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 27.906050625s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [.....XXXXX....X.............X.X.............................] duration: 3.726036745s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 14:10:58 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.72606043s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://version.storj.io/"} failure (antiblip window 3/60): (http.version_storj_io) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000583807s target: https://version.storj.io/ trace: https://jaeger.dev.storj.io/trace/d4412028e1c1a58 FAIL (2026-05-14 14:10:55 UTC) orbiter warning: Get "https://version.storj.io/": dial tcp 35.209.24.65:443: i/o timeout Duration: 30.000598595s core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [......XXXXX....X.............X.X............................] duration: 3.880144471s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 14:10:26 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.880189477s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..........XX............X...................................] duration: 6.699175003s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 14:10:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.699261428s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...........XX............X..................................] duration: 14.076899382s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 14:10:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 14.076917786s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X......X..............................................] duration: 6.382814925s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 14:10:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.382841786s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [.......XXXXX....X.............X.X...........................] duration: 3.942361524s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 14:09:20 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.942387535s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [............XX............X.................................] duration: 6.591191092s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 14:09:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.591212373s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X......X.............................................] duration: 5.957648993s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 14:09:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.957670505s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [........XXXXX....X.............X.X..........................] duration: 3.995994879s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 14:08:58 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.996098646s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X......X............................................] duration: 6.965008805s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 14:08:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.965035625s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.............XX............X................................] duration: 6.789566672s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 14:08:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.789588233s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [.........XXXXX....X.............X.X.........................] duration: 3.957543299s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 14:08:06 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.957567274s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..............XX............X...............................] duration: 6.704932086s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 14:08:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.704997251s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X......X...........................................] duration: 6.493910775s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 14:07:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.493938548s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [..........XXXXX....X.............X.X........................] duration: 3.880852814s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 14:07:17 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.880876809s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X......X..........................................] duration: 6.371416992s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 14:07:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.371433323s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...............XX............X..............................] duration: 6.667443721s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 14:07:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.667471665s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X......X.........................................] duration: 6.656577775s trace: https://jaeger.dev.storj.io/trace/6bcf8e4c5b0862a8 FAIL (2026-05-14 14:06:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== May 14 13:36:28.324422154 (330h48m38.845245899s) [upload=316975]: Starting upload May 14 13:36:28.324487729 (330h48m38.845311474s) [upload=316975]: Done starting upload May 14 13:36:28.324543405 (330h48m38.84536714s) [upload=316975]: Uploading segments... May 14 13:36:28.324757794 (330h48m38.845581539s) [upload=316975]: Got next segment. Inline: false May 14 13:36:28.324788633 (330h48m38.845612368s) [upload=316975]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:36:28.324792991 (330h48m38.845616726s) [upload=316975]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:36:28.567718597 (330h48m39.088542342s) [seg_pos={0 0}, upload=316975]: Begin upload segment... May 14 13:36:28.567770777 (330h48m39.088594522s) [seg_pos={0 0}, upload=316975]: Done begin upload segment. May 14 13:36:28.567794703 (330h48m39.088618438s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.567819981 (330h48m39.088643726s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568338951 (330h48m39.089162696s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.568829707 (330h48m39.089653452s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.569727691 (330h48m39.090551436s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570335932 (330h48m39.091159677s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570398291 (330h48m39.091222026s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570698164 (330h48m39.091521899s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.570954873 (330h48m39.091778618s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571296846 (330h48m39.092120581s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.571662814 (330h48m39.092486549s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572289079 (330h48m39.093112825s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.572752583 (330h48m39.093576329s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573191752 (330h48m39.094015497s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573295591 (330h48m39.094119336s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:28.573700533 (330h48m39.094524288s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Uploading piece... May 14 13:36:29.885678690 (330h48m40.406502435s) [upload=316975]: Next returned nil segment May 14 13:36:29.885693760 (330h48m40.406517505s) [upload=316975]: Waiting for error group managing segments... May 14 13:36:30.267895418 (330h48m40.788719153s) [noise=true , address=167.17.72.69:4716 , piece_id=M6J6OVFWROXK2ZXBAUSUUZJIBGW4QP2NUPG5DSRDYJXIHTJX4T3Q, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.436072812 (330h48m40.956896557s) [noise=true , address=167.17.72.70:4762 , piece_id=3AXWO654HLI5AHEKIH4BR4EH6DQMLOIZR5MLY7JQYCGU6LGGRWYQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.463303156 (330h48m40.984126901s) [noise=true , address=145.79.227.17:4979 , piece_id=HLHTULWWCSGVL6D3VPPMLI5OPCO4YHLWAIBNREXTOEMPMISXX3QQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.495900862 (330h48m41.016724608s) [noise=true , address=77.90.31.10:4542 , piece_id=6CLU53SWUWW7FA5WWNZ6ZWCZYD4KHAP4NUJJXOQZOPTOPIP4H6OA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.545724080 (330h48m41.066547825s) [noise=true , address=77.90.31.12:4618 , piece_id=4R6C7VKGNHOSPBKEK6OJYR5PNC6AX4UBO7WWSANGMPZNWL4J4BEA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.796939386 (330h48m41.317763121s) [noise=true , address=193.149.164.78:4486 , piece_id=Y7PGNX37MO3WAMXLPRWW3IIMPXYVZPUEKHKD27LCKEI5XBJLCSBQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.820623042 (330h48m41.341446847s) [noise=true , address=167.17.72.71:4812 , piece_id=4OGK2UKIDBJC77LQFJGWIETDMSATWDLIOK66BYPMM7QJX3AW2K6A, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:30.914048005 (330h48m41.43487174s) [noise=true , address=167.17.72.68:4663 , piece_id=OWKX55OXLPEABO3YSWV4WPCEEQMYHYY7BIRIXDM2XBSAQEM6B6EQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:31.417499576 (330h48m41.938323331s) [noise=true , address=193.149.164.79:4517 , piece_id=I63XYMJ5KIW4KZRMZKMPKZS76X7HHL7S3BQX4PULEKU73UOPYWZA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:36:32.451371893 (330h48m42.972195648s) [noise=true , address=145.79.227.31:4816 , piece_id=VQBEZTKNZAX2X7ESTZOM2INQ3FENU7OPNCKLAEZ5UI52TXDUIGIA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: <nil> May 14 13:37:27.674200183 (330h49m38.195023918s) [noise=true , address=193.149.164.67:4015 , piece_id=QQ6ASSWTQUEIXV6UHG2IGTP5ETBJAFHGBLD2YYZGHMPYOU67VMLA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12LPgzYrzGJTay9LE1ZNm3dv9vhtj4qQqK5gMdc8B9nyBXdYLuw, address:193.149.164.67:4015): protocol: rpc: dial tcp 193.149.164.67:4015: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674447916 (330h49m38.195271701s) [noise=true , address=193.149.164.69:4070 , piece_id=5IXZR4L6IAX32TRWBFVABDQBSMHA36WNFGCSDBVWBEE5ZBGUR2LA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1xK3trpsZJrYRGZteFknm89ykujQsEnjdYqTXYbWJEK4yp8Wio, address:193.149.164.69:4070): protocol: rpc: dial tcp 193.149.164.69:4070: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674557244 (330h49m38.195380989s) [noise=true , address=193.149.164.66:4988 , piece_id=B7CH3TDCEHFRGAZCWN3OCZJBIPGV2B3V2HC5AJLMLWCJJDR7C2FQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12sCyZhtEDaFcQfcCsETubryphHfHcWzYay5pYMP5xhCuzTUaSd, address:193.149.164.66:4988): protocol: rpc: dial tcp 193.149.164.66:4988: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.674850474 (330h49m38.195674219s) [noise=true , address=193.149.164.71:4310 , piece_id=2OLPYNQE7X6HS4YFJAYEFUG7VVCZN3QV37PHZTJF6S3LREQ6LMQA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:1Ce95mJEDhVmLMjxKY6giHtJDG9gduHXDoNmvZRpmFPAZZTmC2, address:193.149.164.71:4310): rpc: dial tcp 193.149.164.71:4310: i/o timeout May 14 13:37:27.674997264 (330h49m38.195821009s) [noise=true , address=193.149.164.70:4108 , piece_id=PKSTYTZFCNOXPAYVB6XOMT3LN5QTTX2QLSX3ECAGSYSBOLPIWSNA, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12aVyaTYczsdh8UPxFVHrcKjxHxz75b3Q67BcivVDLtM3beaSLm, address:193.149.164.70:4108): protocol: write tcp 188.166.225.170:41018->193.149.164.70:4108: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:37:27.675143844 (330h49m38.195967589s) [noise=true , address=193.149.164.68:4060 , piece_id=WZDIN7U62DMQLPWRLGHEJAMU4POSK7QSDW3FX54CH3LVP7P2RMZQ, seg_pos={0 0}, upload=316975]: Done uploading piece. err: ecclient: upload failed (node:12UEesqk1Bez6iG4eUgV5Kpy1mM2zFmA3bVgCU4w5jF4KuX8DQP, address:193.149.164.68:4060): protocol: rpc: dial tcp 193.149.164.68:4060: i/o timeout; context deadline exceeded May 14 13:37:27.675178129 (330h49m38.196001874s) [seg_pos={0 0}, upload=316975]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> May 14 13:37:27.920638022 (330h49m38.441461777s) [upload=316975]: Done uploading segments... May 14 13:37:27.920680102 (330h49m38.441503907s) [upload=316975]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.656602351s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [................XX............X.............................] duration: 6.737897173s trace: https://jaeger.dev.storj.io/trace/73b73ed5bcf2ff43 FAIL (2026-05-14 14:06:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.73792261s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 8 out of 60 is failed (threshold=6) [...........XXXXX....X.............X.X.......................] duration: 3.766787703s trace: https://jaeger.dev.storj.io/trace/273a3254470045aa FAIL (2026-05-14 14:06:22 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": context deadline exceeded Duration: 3.766809856s core uplink us-select-1 us1-placement-20

1
...
1879
1880
1881
1882
1883
...
16845