Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 9.931247912s trace: https://jaeger.dev.storj.io/trace/3a29744576227984 FAIL (2026-05-14 13:33:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== May 14 13:30:47.226200707 (330h42m57.747024492s) [upload=316880]: Starting upload May 14 13:30:47.226319373 (330h42m57.747143118s) [upload=316880]: Done starting upload May 14 13:30:47.226358969 (330h42m57.747182714s) [upload=316880]: Uploading segments... May 14 13:30:47.226636899 (330h42m57.747460644s) [upload=316880]: Got next segment. Inline: false May 14 13:30:47.226697074 (330h42m57.747520869s) [upload=316880]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:30:47.226703506 (330h42m57.747527301s) [upload=316880]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:30:47.465525228 (330h42m57.986349044s) [seg_pos={0 0}, upload=316880]: Begin upload segment... May 14 13:30:47.465590833 (330h42m57.986414578s) [seg_pos={0 0}, upload=316880]: Done begin upload segment. May 14 13:30:47.465637453 (330h42m57.986474753s) [noise=true , address=193.149.164.69:4086 , piece_id=G6F4G6RYFL2LKLUU5BK4C6YFVJFTGDN46QGYTCR6ZJILB2QAKBQQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.466404867 (330h42m57.987228602s) [noise=true , address=145.79.227.52:4469 , piece_id=HOOUCU52BETFERFHQHRGADFS5CXNPSKDBRUYOR5DHV6YIYZSWZFQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.466633984 (330h42m57.987457729s) [noise=true , address=193.149.164.79:4535 , piece_id=MUMSFNL4A7UURMP4S52E6QDKEMZPWQCBMIARVU7REW7ZPJIGYRPA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467049557 (330h42m57.987873292s) [noise=true , address=77.90.31.14:4685 , piece_id=JKRHMFHPAFDG5DYSOMZDMMUTMKEFMYAZAQ2FSVANPR7ZEAPOP5RA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467363767 (330h42m57.988187512s) [noise=true , address=193.149.164.78:4472 , piece_id=3ZMQGKCDOWN6KF7NDG4Q2WGQZN64HKZLBDFQXAEAS7FNCEYJE4HQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467398434 (330h42m57.988222179s) [noise=true , address=145.79.227.8:4340 , piece_id=I735YQSV2BFCRM3IN4YAOPUMXQHPFNFVIF3OZMWZO47JD55P3EIA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467667005 (330h42m57.98849075s) [noise=true , address=167.17.72.71:4802 , piece_id=EG67URQ6BSLB72EX6VXDNN4KWMF6JF4JMTMQDCKZODIJENM2H4EA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.468006253 (330h42m57.988829988s) [noise=true , address=193.149.164.70:4128 , piece_id=YQL4CPV6AP3GXDQXPG3PUNJYBSTD7A3KMZDGTJBAJTVBTE777YUQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.468130280 (330h42m57.988954035s) [noise=true , address=45.91.116.14:4203 , piece_id=O7QHYQI4KAK7VAWXH4BALEXGH5JRVWDFGRFJ72MN3FDJ5UVQZIVQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.468822761 (330h42m57.989646506s) [noise=true , address=145.79.227.44:4215 , piece_id=6SWT5RS6AVXU3YQ6YM4OFSEUFMYCJHVU75GEFAF4BT5QHSJ424YQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.469344106 (330h42m57.990167861s) [noise=true , address=193.149.164.71:4300 , piece_id=XUV6Q44FFLERGHWPKH3LTZJXLTB7K5TVJTBXU34LSWPDYCGEVAIQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.469588653 (330h42m57.990412398s) [noise=true , address=193.149.164.67:4030 , piece_id=XZOKQ2HW7XG2RKWPL3C7RJS2LJAC6XJHT7XLOJSQ74MEBDI2FCMQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.470155565 (330h42m57.99097931s) [noise=true , address=167.17.72.70:4772 , piece_id=6BE73P7NXYUYTLIXTAYRAY3AL5YXWAH6US325MLNE5HWLQY2DXGA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.471342059 (330h42m57.992165804s) [noise=true , address=145.79.227.30:4782 , piece_id=ZHAOTMWKAPLBQ7V2XNGVGRXW25A4GKOBZ2SBI4ZFKANJ7VE3IOJQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.471793761 (330h42m57.992617506s) [noise=true , address=167.17.72.69:4740 , piece_id=UYGNMGOUAOHJQGAUSUJF2SKB23Z2BXOL24DTIICWFAF46EJKULRA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.472818255 (330h42m57.99364201s) [noise=true , address=45.91.116.10:4086 , piece_id=5NP3LBLYO4KULELOHPO6KWSZPZUW5WWFD6GK64PHR6L2QJ7TEG4Q, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:48.824555337 (330h42m59.345379072s) [upload=316880]: Next returned nil segment May 14 13:30:48.824567069 (330h42m59.345390814s) [upload=316880]: Waiting for error group managing segments... May 14 13:30:49.208040236 (330h42m59.728863981s) [noise=true , address=167.17.72.71:4802 , piece_id=EG67URQ6BSLB72EX6VXDNN4KWMF6JF4JMTMQDCKZODIJENM2H4EA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.354760988 (330h42m59.875584734s) [noise=true , address=145.79.227.30:4782 , piece_id=ZHAOTMWKAPLBQ7V2XNGVGRXW25A4GKOBZ2SBI4ZFKANJ7VE3IOJQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.355223522 (330h42m59.876047257s) [noise=true , address=145.79.227.8:4340 , piece_id=I735YQSV2BFCRM3IN4YAOPUMXQHPFNFVIF3OZMWZO47JD55P3EIA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.430937189 (330h42m59.951760934s) [noise=true , address=145.79.227.52:4469 , piece_id=HOOUCU52BETFERFHQHRGADFS5CXNPSKDBRUYOR5DHV6YIYZSWZFQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.455996030 (330h42m59.976819775s) [noise=true , address=77.90.31.14:4685 , piece_id=JKRHMFHPAFDG5DYSOMZDMMUTMKEFMYAZAQ2FSVANPR7ZEAPOP5RA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.954832519 (330h43m0.475656264s) [noise=true , address=145.79.227.44:4215 , piece_id=6SWT5RS6AVXU3YQ6YM4OFSEUFMYCJHVU75GEFAF4BT5QHSJ424YQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.989917582 (330h43m0.510741327s) [noise=true , address=45.91.116.10:4086 , piece_id=5NP3LBLYO4KULELOHPO6KWSZPZUW5WWFD6GK64PHR6L2QJ7TEG4Q, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:50.082006168 (330h43m0.602829913s) [noise=true , address=45.91.116.14:4203 , piece_id=O7QHYQI4KAK7VAWXH4BALEXGH5JRVWDFGRFJ72MN3FDJ5UVQZIVQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:53.596731701 (330h43m4.117555446s) [noise=true , address=193.149.164.79:4535 , piece_id=MUMSFNL4A7UURMP4S52E6QDKEMZPWQCBMIARVU7REW7ZPJIGYRPA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:55.497462951 (330h43m6.018286696s) [noise=true , address=167.17.72.70:4772 , piece_id=6BE73P7NXYUYTLIXTAYRAY3AL5YXWAH6US325MLNE5HWLQY2DXGA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:56.403300485 (330h43m6.92412423s) [noise=true , address=167.17.72.69:4740 , piece_id=UYGNMGOUAOHJQGAUSUJF2SKB23Z2BXOL24DTIICWFAF46EJKULRA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:59.324401701 (330h43m9.845225446s) [noise=true , address=193.149.164.78:4472 , piece_id=3ZMQGKCDOWN6KF7NDG4Q2WGQZN64HKZLBDFQXAEAS7FNCEYJE4HQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:31:46.566908056 (330h43m57.087731801s) [noise=true , address=193.149.164.70:4128 , piece_id=YQL4CPV6AP3GXDQXPG3PUNJYBSTD7A3KMZDGTJBAJTVBTE777YUQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:1286axL8SEBRohFVBHVcMGmAc4ktdJQg37LxrknzonqtXWFXiLF, address:193.149.164.70:4128): protocol: write tcp 188.166.225.170:54916->193.149.164.70:4128: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:31:46.566940518 (330h43m57.087764253s) [noise=true , address=193.149.164.67:4030 , piece_id=XZOKQ2HW7XG2RKWPL3C7RJS2LJAC6XJHT7XLOJSQ74MEBDI2FCMQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:1Xs7CkcBfkunXRiLxfEU3KWt497dpskeJJoJTeAeXjesgUN7s8, address:193.149.164.67:4030): protocol: rpc: dial tcp 193.149.164.67:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:31:46.567027874 (330h43m57.087851619s) [noise=true , address=193.149.164.69:4086 , piece_id=G6F4G6RYFL2LKLUU5BK4C6YFVJFTGDN46QGYTCR6ZJILB2QAKBQQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:12kBkPaBtFwEHVi9Ly4WQNGoWMFYa7dHujcJ1GKdqQ4t8vzR9w5, address:193.149.164.69:4086): protocol: write tcp 188.166.225.170:38382->193.149.164.69:4086: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:31:46.567148054 (330h43m57.087971799s) [noise=true , address=193.149.164.71:4300 , piece_id=XUV6Q44FFLERGHWPKH3LTZJXLTB7K5TVJTBXU34LSWPDYCGEVAIQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:12GRF8go19rLx7rM73XqiV1bBvXaYz3kf8rgfFFA3shmYnkAKZ9, address:193.149.164.71:4300): rpc: dial tcp 193.149.164.71:4300: i/o timeout May 14 13:31:46.567179203 (330h43m57.088002948s) [seg_pos={0 0}, upload=316880]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> May 14 13:31:46.811323383 (330h43m57.332147118s) [upload=316880]: Done uploading segments... May 14 13:31:46.811354222 (330h43m57.332177957s) [upload=316880]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 9.931270133s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [....................................X..................XX...] duration: 7.185017874s trace: https://jaeger.dev.storj.io/trace/4edc8e7010a29a3f FAIL (2026-05-14 13:33:45 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.185028504s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [......................................................XXXXX.] duration: 4.191130292s trace: https://jaeger.dev.storj.io/trace/50df6e7df0ed4adf FAIL (2026-05-14 13:33:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.191154988s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 7.568083025s trace: https://jaeger.dev.storj.io/trace/3a29744576227984 FAIL (2026-05-14 13:33:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== May 14 13:30:47.226200707 (330h42m57.747024492s) [upload=316880]: Starting upload May 14 13:30:47.226319373 (330h42m57.747143118s) [upload=316880]: Done starting upload May 14 13:30:47.226358969 (330h42m57.747182714s) [upload=316880]: Uploading segments... May 14 13:30:47.226636899 (330h42m57.747460644s) [upload=316880]: Got next segment. Inline: false May 14 13:30:47.226697074 (330h42m57.747520869s) [upload=316880]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:30:47.226703506 (330h42m57.747527301s) [upload=316880]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:30:47.465525228 (330h42m57.986349044s) [seg_pos={0 0}, upload=316880]: Begin upload segment... May 14 13:30:47.465590833 (330h42m57.986414578s) [seg_pos={0 0}, upload=316880]: Done begin upload segment. May 14 13:30:47.465637453 (330h42m57.986474753s) [noise=true , address=193.149.164.69:4086 , piece_id=G6F4G6RYFL2LKLUU5BK4C6YFVJFTGDN46QGYTCR6ZJILB2QAKBQQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.466404867 (330h42m57.987228602s) [noise=true , address=145.79.227.52:4469 , piece_id=HOOUCU52BETFERFHQHRGADFS5CXNPSKDBRUYOR5DHV6YIYZSWZFQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.466633984 (330h42m57.987457729s) [noise=true , address=193.149.164.79:4535 , piece_id=MUMSFNL4A7UURMP4S52E6QDKEMZPWQCBMIARVU7REW7ZPJIGYRPA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467049557 (330h42m57.987873292s) [noise=true , address=77.90.31.14:4685 , piece_id=JKRHMFHPAFDG5DYSOMZDMMUTMKEFMYAZAQ2FSVANPR7ZEAPOP5RA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467363767 (330h42m57.988187512s) [noise=true , address=193.149.164.78:4472 , piece_id=3ZMQGKCDOWN6KF7NDG4Q2WGQZN64HKZLBDFQXAEAS7FNCEYJE4HQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467398434 (330h42m57.988222179s) [noise=true , address=145.79.227.8:4340 , piece_id=I735YQSV2BFCRM3IN4YAOPUMXQHPFNFVIF3OZMWZO47JD55P3EIA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467667005 (330h42m57.98849075s) [noise=true , address=167.17.72.71:4802 , piece_id=EG67URQ6BSLB72EX6VXDNN4KWMF6JF4JMTMQDCKZODIJENM2H4EA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.468006253 (330h42m57.988829988s) [noise=true , address=193.149.164.70:4128 , piece_id=YQL4CPV6AP3GXDQXPG3PUNJYBSTD7A3KMZDGTJBAJTVBTE777YUQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.468130280 (330h42m57.988954035s) [noise=true , address=45.91.116.14:4203 , piece_id=O7QHYQI4KAK7VAWXH4BALEXGH5JRVWDFGRFJ72MN3FDJ5UVQZIVQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.468822761 (330h42m57.989646506s) [noise=true , address=145.79.227.44:4215 , piece_id=6SWT5RS6AVXU3YQ6YM4OFSEUFMYCJHVU75GEFAF4BT5QHSJ424YQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.469344106 (330h42m57.990167861s) [noise=true , address=193.149.164.71:4300 , piece_id=XUV6Q44FFLERGHWPKH3LTZJXLTB7K5TVJTBXU34LSWPDYCGEVAIQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.469588653 (330h42m57.990412398s) [noise=true , address=193.149.164.67:4030 , piece_id=XZOKQ2HW7XG2RKWPL3C7RJS2LJAC6XJHT7XLOJSQ74MEBDI2FCMQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.470155565 (330h42m57.99097931s) [noise=true , address=167.17.72.70:4772 , piece_id=6BE73P7NXYUYTLIXTAYRAY3AL5YXWAH6US325MLNE5HWLQY2DXGA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.471342059 (330h42m57.992165804s) [noise=true , address=145.79.227.30:4782 , piece_id=ZHAOTMWKAPLBQ7V2XNGVGRXW25A4GKOBZ2SBI4ZFKANJ7VE3IOJQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.471793761 (330h42m57.992617506s) [noise=true , address=167.17.72.69:4740 , piece_id=UYGNMGOUAOHJQGAUSUJF2SKB23Z2BXOL24DTIICWFAF46EJKULRA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.472818255 (330h42m57.99364201s) [noise=true , address=45.91.116.10:4086 , piece_id=5NP3LBLYO4KULELOHPO6KWSZPZUW5WWFD6GK64PHR6L2QJ7TEG4Q, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:48.824555337 (330h42m59.345379072s) [upload=316880]: Next returned nil segment May 14 13:30:48.824567069 (330h42m59.345390814s) [upload=316880]: Waiting for error group managing segments... May 14 13:30:49.208040236 (330h42m59.728863981s) [noise=true , address=167.17.72.71:4802 , piece_id=EG67URQ6BSLB72EX6VXDNN4KWMF6JF4JMTMQDCKZODIJENM2H4EA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.354760988 (330h42m59.875584734s) [noise=true , address=145.79.227.30:4782 , piece_id=ZHAOTMWKAPLBQ7V2XNGVGRXW25A4GKOBZ2SBI4ZFKANJ7VE3IOJQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.355223522 (330h42m59.876047257s) [noise=true , address=145.79.227.8:4340 , piece_id=I735YQSV2BFCRM3IN4YAOPUMXQHPFNFVIF3OZMWZO47JD55P3EIA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.430937189 (330h42m59.951760934s) [noise=true , address=145.79.227.52:4469 , piece_id=HOOUCU52BETFERFHQHRGADFS5CXNPSKDBRUYOR5DHV6YIYZSWZFQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.455996030 (330h42m59.976819775s) [noise=true , address=77.90.31.14:4685 , piece_id=JKRHMFHPAFDG5DYSOMZDMMUTMKEFMYAZAQ2FSVANPR7ZEAPOP5RA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.954832519 (330h43m0.475656264s) [noise=true , address=145.79.227.44:4215 , piece_id=6SWT5RS6AVXU3YQ6YM4OFSEUFMYCJHVU75GEFAF4BT5QHSJ424YQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.989917582 (330h43m0.510741327s) [noise=true , address=45.91.116.10:4086 , piece_id=5NP3LBLYO4KULELOHPO6KWSZPZUW5WWFD6GK64PHR6L2QJ7TEG4Q, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:50.082006168 (330h43m0.602829913s) [noise=true , address=45.91.116.14:4203 , piece_id=O7QHYQI4KAK7VAWXH4BALEXGH5JRVWDFGRFJ72MN3FDJ5UVQZIVQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:53.596731701 (330h43m4.117555446s) [noise=true , address=193.149.164.79:4535 , piece_id=MUMSFNL4A7UURMP4S52E6QDKEMZPWQCBMIARVU7REW7ZPJIGYRPA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:55.497462951 (330h43m6.018286696s) [noise=true , address=167.17.72.70:4772 , piece_id=6BE73P7NXYUYTLIXTAYRAY3AL5YXWAH6US325MLNE5HWLQY2DXGA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:56.403300485 (330h43m6.92412423s) [noise=true , address=167.17.72.69:4740 , piece_id=UYGNMGOUAOHJQGAUSUJF2SKB23Z2BXOL24DTIICWFAF46EJKULRA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:59.324401701 (330h43m9.845225446s) [noise=true , address=193.149.164.78:4472 , piece_id=3ZMQGKCDOWN6KF7NDG4Q2WGQZN64HKZLBDFQXAEAS7FNCEYJE4HQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:31:46.566908056 (330h43m57.087731801s) [noise=true , address=193.149.164.70:4128 , piece_id=YQL4CPV6AP3GXDQXPG3PUNJYBSTD7A3KMZDGTJBAJTVBTE777YUQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:1286axL8SEBRohFVBHVcMGmAc4ktdJQg37LxrknzonqtXWFXiLF, address:193.149.164.70:4128): protocol: write tcp 188.166.225.170:54916->193.149.164.70:4128: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:31:46.566940518 (330h43m57.087764253s) [noise=true , address=193.149.164.67:4030 , piece_id=XZOKQ2HW7XG2RKWPL3C7RJS2LJAC6XJHT7XLOJSQ74MEBDI2FCMQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:1Xs7CkcBfkunXRiLxfEU3KWt497dpskeJJoJTeAeXjesgUN7s8, address:193.149.164.67:4030): protocol: rpc: dial tcp 193.149.164.67:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:31:46.567027874 (330h43m57.087851619s) [noise=true , address=193.149.164.69:4086 , piece_id=G6F4G6RYFL2LKLUU5BK4C6YFVJFTGDN46QGYTCR6ZJILB2QAKBQQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:12kBkPaBtFwEHVi9Ly4WQNGoWMFYa7dHujcJ1GKdqQ4t8vzR9w5, address:193.149.164.69:4086): protocol: write tcp 188.166.225.170:38382->193.149.164.69:4086: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:31:46.567148054 (330h43m57.087971799s) [noise=true , address=193.149.164.71:4300 , piece_id=XUV6Q44FFLERGHWPKH3LTZJXLTB7K5TVJTBXU34LSWPDYCGEVAIQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:12GRF8go19rLx7rM73XqiV1bBvXaYz3kf8rgfFFA3shmYnkAKZ9, address:193.149.164.71:4300): rpc: dial tcp 193.149.164.71:4300: i/o timeout May 14 13:31:46.567179203 (330h43m57.088002948s) [seg_pos={0 0}, upload=316880]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> May 14 13:31:46.811323383 (330h43m57.332147118s) [upload=316880]: Done uploading segments... May 14 13:31:46.811354222 (330h43m57.332177957s) [upload=316880]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 7.568099586s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.....................................X..................XX..] duration: 13.366704171s trace: https://jaeger.dev.storj.io/trace/4edc8e7010a29a3f FAIL (2026-05-14 13:33:15 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 13.36672414s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 33.566429408s trace: https://jaeger.dev.storj.io/trace/3a29744576227984 FAIL (2026-05-14 13:33:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== May 14 13:30:47.226200707 (330h42m57.747024492s) [upload=316880]: Starting upload May 14 13:30:47.226319373 (330h42m57.747143118s) [upload=316880]: Done starting upload May 14 13:30:47.226358969 (330h42m57.747182714s) [upload=316880]: Uploading segments... May 14 13:30:47.226636899 (330h42m57.747460644s) [upload=316880]: Got next segment. Inline: false May 14 13:30:47.226697074 (330h42m57.747520869s) [upload=316880]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:30:47.226703506 (330h42m57.747527301s) [upload=316880]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:30:47.465525228 (330h42m57.986349044s) [seg_pos={0 0}, upload=316880]: Begin upload segment... May 14 13:30:47.465590833 (330h42m57.986414578s) [seg_pos={0 0}, upload=316880]: Done begin upload segment. May 14 13:30:47.465637453 (330h42m57.986474753s) [noise=true , address=193.149.164.69:4086 , piece_id=G6F4G6RYFL2LKLUU5BK4C6YFVJFTGDN46QGYTCR6ZJILB2QAKBQQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.466404867 (330h42m57.987228602s) [noise=true , address=145.79.227.52:4469 , piece_id=HOOUCU52BETFERFHQHRGADFS5CXNPSKDBRUYOR5DHV6YIYZSWZFQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.466633984 (330h42m57.987457729s) [noise=true , address=193.149.164.79:4535 , piece_id=MUMSFNL4A7UURMP4S52E6QDKEMZPWQCBMIARVU7REW7ZPJIGYRPA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467049557 (330h42m57.987873292s) [noise=true , address=77.90.31.14:4685 , piece_id=JKRHMFHPAFDG5DYSOMZDMMUTMKEFMYAZAQ2FSVANPR7ZEAPOP5RA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467363767 (330h42m57.988187512s) [noise=true , address=193.149.164.78:4472 , piece_id=3ZMQGKCDOWN6KF7NDG4Q2WGQZN64HKZLBDFQXAEAS7FNCEYJE4HQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467398434 (330h42m57.988222179s) [noise=true , address=145.79.227.8:4340 , piece_id=I735YQSV2BFCRM3IN4YAOPUMXQHPFNFVIF3OZMWZO47JD55P3EIA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467667005 (330h42m57.98849075s) [noise=true , address=167.17.72.71:4802 , piece_id=EG67URQ6BSLB72EX6VXDNN4KWMF6JF4JMTMQDCKZODIJENM2H4EA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.468006253 (330h42m57.988829988s) [noise=true , address=193.149.164.70:4128 , piece_id=YQL4CPV6AP3GXDQXPG3PUNJYBSTD7A3KMZDGTJBAJTVBTE777YUQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.468130280 (330h42m57.988954035s) [noise=true , address=45.91.116.14:4203 , piece_id=O7QHYQI4KAK7VAWXH4BALEXGH5JRVWDFGRFJ72MN3FDJ5UVQZIVQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.468822761 (330h42m57.989646506s) [noise=true , address=145.79.227.44:4215 , piece_id=6SWT5RS6AVXU3YQ6YM4OFSEUFMYCJHVU75GEFAF4BT5QHSJ424YQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.469344106 (330h42m57.990167861s) [noise=true , address=193.149.164.71:4300 , piece_id=XUV6Q44FFLERGHWPKH3LTZJXLTB7K5TVJTBXU34LSWPDYCGEVAIQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.469588653 (330h42m57.990412398s) [noise=true , address=193.149.164.67:4030 , piece_id=XZOKQ2HW7XG2RKWPL3C7RJS2LJAC6XJHT7XLOJSQ74MEBDI2FCMQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.470155565 (330h42m57.99097931s) [noise=true , address=167.17.72.70:4772 , piece_id=6BE73P7NXYUYTLIXTAYRAY3AL5YXWAH6US325MLNE5HWLQY2DXGA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.471342059 (330h42m57.992165804s) [noise=true , address=145.79.227.30:4782 , piece_id=ZHAOTMWKAPLBQ7V2XNGVGRXW25A4GKOBZ2SBI4ZFKANJ7VE3IOJQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.471793761 (330h42m57.992617506s) [noise=true , address=167.17.72.69:4740 , piece_id=UYGNMGOUAOHJQGAUSUJF2SKB23Z2BXOL24DTIICWFAF46EJKULRA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.472818255 (330h42m57.99364201s) [noise=true , address=45.91.116.10:4086 , piece_id=5NP3LBLYO4KULELOHPO6KWSZPZUW5WWFD6GK64PHR6L2QJ7TEG4Q, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:48.824555337 (330h42m59.345379072s) [upload=316880]: Next returned nil segment May 14 13:30:48.824567069 (330h42m59.345390814s) [upload=316880]: Waiting for error group managing segments... May 14 13:30:49.208040236 (330h42m59.728863981s) [noise=true , address=167.17.72.71:4802 , piece_id=EG67URQ6BSLB72EX6VXDNN4KWMF6JF4JMTMQDCKZODIJENM2H4EA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.354760988 (330h42m59.875584734s) [noise=true , address=145.79.227.30:4782 , piece_id=ZHAOTMWKAPLBQ7V2XNGVGRXW25A4GKOBZ2SBI4ZFKANJ7VE3IOJQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.355223522 (330h42m59.876047257s) [noise=true , address=145.79.227.8:4340 , piece_id=I735YQSV2BFCRM3IN4YAOPUMXQHPFNFVIF3OZMWZO47JD55P3EIA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.430937189 (330h42m59.951760934s) [noise=true , address=145.79.227.52:4469 , piece_id=HOOUCU52BETFERFHQHRGADFS5CXNPSKDBRUYOR5DHV6YIYZSWZFQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.455996030 (330h42m59.976819775s) [noise=true , address=77.90.31.14:4685 , piece_id=JKRHMFHPAFDG5DYSOMZDMMUTMKEFMYAZAQ2FSVANPR7ZEAPOP5RA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.954832519 (330h43m0.475656264s) [noise=true , address=145.79.227.44:4215 , piece_id=6SWT5RS6AVXU3YQ6YM4OFSEUFMYCJHVU75GEFAF4BT5QHSJ424YQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.989917582 (330h43m0.510741327s) [noise=true , address=45.91.116.10:4086 , piece_id=5NP3LBLYO4KULELOHPO6KWSZPZUW5WWFD6GK64PHR6L2QJ7TEG4Q, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:50.082006168 (330h43m0.602829913s) [noise=true , address=45.91.116.14:4203 , piece_id=O7QHYQI4KAK7VAWXH4BALEXGH5JRVWDFGRFJ72MN3FDJ5UVQZIVQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:53.596731701 (330h43m4.117555446s) [noise=true , address=193.149.164.79:4535 , piece_id=MUMSFNL4A7UURMP4S52E6QDKEMZPWQCBMIARVU7REW7ZPJIGYRPA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:55.497462951 (330h43m6.018286696s) [noise=true , address=167.17.72.70:4772 , piece_id=6BE73P7NXYUYTLIXTAYRAY3AL5YXWAH6US325MLNE5HWLQY2DXGA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:56.403300485 (330h43m6.92412423s) [noise=true , address=167.17.72.69:4740 , piece_id=UYGNMGOUAOHJQGAUSUJF2SKB23Z2BXOL24DTIICWFAF46EJKULRA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:59.324401701 (330h43m9.845225446s) [noise=true , address=193.149.164.78:4472 , piece_id=3ZMQGKCDOWN6KF7NDG4Q2WGQZN64HKZLBDFQXAEAS7FNCEYJE4HQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:31:46.566908056 (330h43m57.087731801s) [noise=true , address=193.149.164.70:4128 , piece_id=YQL4CPV6AP3GXDQXPG3PUNJYBSTD7A3KMZDGTJBAJTVBTE777YUQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:1286axL8SEBRohFVBHVcMGmAc4ktdJQg37LxrknzonqtXWFXiLF, address:193.149.164.70:4128): protocol: write tcp 188.166.225.170:54916->193.149.164.70:4128: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:31:46.566940518 (330h43m57.087764253s) [noise=true , address=193.149.164.67:4030 , piece_id=XZOKQ2HW7XG2RKWPL3C7RJS2LJAC6XJHT7XLOJSQ74MEBDI2FCMQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:1Xs7CkcBfkunXRiLxfEU3KWt497dpskeJJoJTeAeXjesgUN7s8, address:193.149.164.67:4030): protocol: rpc: dial tcp 193.149.164.67:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:31:46.567027874 (330h43m57.087851619s) [noise=true , address=193.149.164.69:4086 , piece_id=G6F4G6RYFL2LKLUU5BK4C6YFVJFTGDN46QGYTCR6ZJILB2QAKBQQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:12kBkPaBtFwEHVi9Ly4WQNGoWMFYa7dHujcJ1GKdqQ4t8vzR9w5, address:193.149.164.69:4086): protocol: write tcp 188.166.225.170:38382->193.149.164.69:4086: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:31:46.567148054 (330h43m57.087971799s) [noise=true , address=193.149.164.71:4300 , piece_id=XUV6Q44FFLERGHWPKH3LTZJXLTB7K5TVJTBXU34LSWPDYCGEVAIQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:12GRF8go19rLx7rM73XqiV1bBvXaYz3kf8rgfFFA3shmYnkAKZ9, address:193.149.164.71:4300): rpc: dial tcp 193.149.164.71:4300: i/o timeout May 14 13:31:46.567179203 (330h43m57.088002948s) [seg_pos={0 0}, upload=316880]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> May 14 13:31:46.811323383 (330h43m57.332147118s) [upload=316880]: Done uploading segments... May 14 13:31:46.811354222 (330h43m57.332177957s) [upload=316880]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 33.566456819s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [.......................................................XXXXX] duration: 5.000522825s trace: https://jaeger.dev.storj.io/trace/50df6e7df0ed4adf FAIL (2026-05-14 13:32:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000539286s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [......................................X..................XX.] duration: 19.842734562s trace: https://jaeger.dev.storj.io/trace/4edc8e7010a29a3f FAIL (2026-05-14 13:32:31 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 19.842750312s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [........................................................XXXX] duration: 5.001555025s trace: https://jaeger.dev.storj.io/trace/70ec87dc7c9c042a FAIL (2026-05-14 13:32:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001573772s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.......................................X..................XX] duration: 1m0.000647718s trace: https://jaeger.dev.storj.io/trace/4edc8e7010a29a3f FAIL (2026-05-14 13:32:11 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.00065906s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 22.157995884s trace: https://jaeger.dev.storj.io/trace/3a29744576227984 FAIL (2026-05-14 13:32:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== May 14 13:30:47.226200707 (330h42m57.747024492s) [upload=316880]: Starting upload May 14 13:30:47.226319373 (330h42m57.747143118s) [upload=316880]: Done starting upload May 14 13:30:47.226358969 (330h42m57.747182714s) [upload=316880]: Uploading segments... May 14 13:30:47.226636899 (330h42m57.747460644s) [upload=316880]: Got next segment. Inline: false May 14 13:30:47.226697074 (330h42m57.747520869s) [upload=316880]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:30:47.226703506 (330h42m57.747527301s) [upload=316880]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:30:47.465525228 (330h42m57.986349044s) [seg_pos={0 0}, upload=316880]: Begin upload segment... May 14 13:30:47.465590833 (330h42m57.986414578s) [seg_pos={0 0}, upload=316880]: Done begin upload segment. May 14 13:30:47.465637453 (330h42m57.986474753s) [noise=true , address=193.149.164.69:4086 , piece_id=G6F4G6RYFL2LKLUU5BK4C6YFVJFTGDN46QGYTCR6ZJILB2QAKBQQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.466404867 (330h42m57.987228602s) [noise=true , address=145.79.227.52:4469 , piece_id=HOOUCU52BETFERFHQHRGADFS5CXNPSKDBRUYOR5DHV6YIYZSWZFQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.466633984 (330h42m57.987457729s) [noise=true , address=193.149.164.79:4535 , piece_id=MUMSFNL4A7UURMP4S52E6QDKEMZPWQCBMIARVU7REW7ZPJIGYRPA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467049557 (330h42m57.987873292s) [noise=true , address=77.90.31.14:4685 , piece_id=JKRHMFHPAFDG5DYSOMZDMMUTMKEFMYAZAQ2FSVANPR7ZEAPOP5RA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467363767 (330h42m57.988187512s) [noise=true , address=193.149.164.78:4472 , piece_id=3ZMQGKCDOWN6KF7NDG4Q2WGQZN64HKZLBDFQXAEAS7FNCEYJE4HQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467398434 (330h42m57.988222179s) [noise=true , address=145.79.227.8:4340 , piece_id=I735YQSV2BFCRM3IN4YAOPUMXQHPFNFVIF3OZMWZO47JD55P3EIA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467667005 (330h42m57.98849075s) [noise=true , address=167.17.72.71:4802 , piece_id=EG67URQ6BSLB72EX6VXDNN4KWMF6JF4JMTMQDCKZODIJENM2H4EA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.468006253 (330h42m57.988829988s) [noise=true , address=193.149.164.70:4128 , piece_id=YQL4CPV6AP3GXDQXPG3PUNJYBSTD7A3KMZDGTJBAJTVBTE777YUQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.468130280 (330h42m57.988954035s) [noise=true , address=45.91.116.14:4203 , piece_id=O7QHYQI4KAK7VAWXH4BALEXGH5JRVWDFGRFJ72MN3FDJ5UVQZIVQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.468822761 (330h42m57.989646506s) [noise=true , address=145.79.227.44:4215 , piece_id=6SWT5RS6AVXU3YQ6YM4OFSEUFMYCJHVU75GEFAF4BT5QHSJ424YQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.469344106 (330h42m57.990167861s) [noise=true , address=193.149.164.71:4300 , piece_id=XUV6Q44FFLERGHWPKH3LTZJXLTB7K5TVJTBXU34LSWPDYCGEVAIQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.469588653 (330h42m57.990412398s) [noise=true , address=193.149.164.67:4030 , piece_id=XZOKQ2HW7XG2RKWPL3C7RJS2LJAC6XJHT7XLOJSQ74MEBDI2FCMQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.470155565 (330h42m57.99097931s) [noise=true , address=167.17.72.70:4772 , piece_id=6BE73P7NXYUYTLIXTAYRAY3AL5YXWAH6US325MLNE5HWLQY2DXGA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.471342059 (330h42m57.992165804s) [noise=true , address=145.79.227.30:4782 , piece_id=ZHAOTMWKAPLBQ7V2XNGVGRXW25A4GKOBZ2SBI4ZFKANJ7VE3IOJQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.471793761 (330h42m57.992617506s) [noise=true , address=167.17.72.69:4740 , piece_id=UYGNMGOUAOHJQGAUSUJF2SKB23Z2BXOL24DTIICWFAF46EJKULRA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.472818255 (330h42m57.99364201s) [noise=true , address=45.91.116.10:4086 , piece_id=5NP3LBLYO4KULELOHPO6KWSZPZUW5WWFD6GK64PHR6L2QJ7TEG4Q, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:48.824555337 (330h42m59.345379072s) [upload=316880]: Next returned nil segment May 14 13:30:48.824567069 (330h42m59.345390814s) [upload=316880]: Waiting for error group managing segments... May 14 13:30:49.208040236 (330h42m59.728863981s) [noise=true , address=167.17.72.71:4802 , piece_id=EG67URQ6BSLB72EX6VXDNN4KWMF6JF4JMTMQDCKZODIJENM2H4EA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.354760988 (330h42m59.875584734s) [noise=true , address=145.79.227.30:4782 , piece_id=ZHAOTMWKAPLBQ7V2XNGVGRXW25A4GKOBZ2SBI4ZFKANJ7VE3IOJQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.355223522 (330h42m59.876047257s) [noise=true , address=145.79.227.8:4340 , piece_id=I735YQSV2BFCRM3IN4YAOPUMXQHPFNFVIF3OZMWZO47JD55P3EIA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.430937189 (330h42m59.951760934s) [noise=true , address=145.79.227.52:4469 , piece_id=HOOUCU52BETFERFHQHRGADFS5CXNPSKDBRUYOR5DHV6YIYZSWZFQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.455996030 (330h42m59.976819775s) [noise=true , address=77.90.31.14:4685 , piece_id=JKRHMFHPAFDG5DYSOMZDMMUTMKEFMYAZAQ2FSVANPR7ZEAPOP5RA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.954832519 (330h43m0.475656264s) [noise=true , address=145.79.227.44:4215 , piece_id=6SWT5RS6AVXU3YQ6YM4OFSEUFMYCJHVU75GEFAF4BT5QHSJ424YQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.989917582 (330h43m0.510741327s) [noise=true , address=45.91.116.10:4086 , piece_id=5NP3LBLYO4KULELOHPO6KWSZPZUW5WWFD6GK64PHR6L2QJ7TEG4Q, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:50.082006168 (330h43m0.602829913s) [noise=true , address=45.91.116.14:4203 , piece_id=O7QHYQI4KAK7VAWXH4BALEXGH5JRVWDFGRFJ72MN3FDJ5UVQZIVQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:53.596731701 (330h43m4.117555446s) [noise=true , address=193.149.164.79:4535 , piece_id=MUMSFNL4A7UURMP4S52E6QDKEMZPWQCBMIARVU7REW7ZPJIGYRPA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:55.497462951 (330h43m6.018286696s) [noise=true , address=167.17.72.70:4772 , piece_id=6BE73P7NXYUYTLIXTAYRAY3AL5YXWAH6US325MLNE5HWLQY2DXGA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:56.403300485 (330h43m6.92412423s) [noise=true , address=167.17.72.69:4740 , piece_id=UYGNMGOUAOHJQGAUSUJF2SKB23Z2BXOL24DTIICWFAF46EJKULRA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:59.324401701 (330h43m9.845225446s) [noise=true , address=193.149.164.78:4472 , piece_id=3ZMQGKCDOWN6KF7NDG4Q2WGQZN64HKZLBDFQXAEAS7FNCEYJE4HQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:31:46.566908056 (330h43m57.087731801s) [noise=true , address=193.149.164.70:4128 , piece_id=YQL4CPV6AP3GXDQXPG3PUNJYBSTD7A3KMZDGTJBAJTVBTE777YUQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:1286axL8SEBRohFVBHVcMGmAc4ktdJQg37LxrknzonqtXWFXiLF, address:193.149.164.70:4128): protocol: write tcp 188.166.225.170:54916->193.149.164.70:4128: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:31:46.566940518 (330h43m57.087764253s) [noise=true , address=193.149.164.67:4030 , piece_id=XZOKQ2HW7XG2RKWPL3C7RJS2LJAC6XJHT7XLOJSQ74MEBDI2FCMQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:1Xs7CkcBfkunXRiLxfEU3KWt497dpskeJJoJTeAeXjesgUN7s8, address:193.149.164.67:4030): protocol: rpc: dial tcp 193.149.164.67:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:31:46.567027874 (330h43m57.087851619s) [noise=true , address=193.149.164.69:4086 , piece_id=G6F4G6RYFL2LKLUU5BK4C6YFVJFTGDN46QGYTCR6ZJILB2QAKBQQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:12kBkPaBtFwEHVi9Ly4WQNGoWMFYa7dHujcJ1GKdqQ4t8vzR9w5, address:193.149.164.69:4086): protocol: write tcp 188.166.225.170:38382->193.149.164.69:4086: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:31:46.567148054 (330h43m57.087971799s) [noise=true , address=193.149.164.71:4300 , piece_id=XUV6Q44FFLERGHWPKH3LTZJXLTB7K5TVJTBXU34LSWPDYCGEVAIQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:12GRF8go19rLx7rM73XqiV1bBvXaYz3kf8rgfFFA3shmYnkAKZ9, address:193.149.164.71:4300): rpc: dial tcp 193.149.164.71:4300: i/o timeout May 14 13:31:46.567179203 (330h43m57.088002948s) [seg_pos={0 0}, upload=316880]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> May 14 13:31:46.811323383 (330h43m57.332147118s) [upload=316880]: Done uploading segments... May 14 13:31:46.811354222 (330h43m57.332177957s) [upload=316880]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 22.158007717s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.245969425s trace: https://jaeger.dev.storj.io/trace/3a29744576227984 FAIL (2026-05-14 13:31:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== May 14 13:30:47.226200707 (330h42m57.747024492s) [upload=316880]: Starting upload May 14 13:30:47.226319373 (330h42m57.747143118s) [upload=316880]: Done starting upload May 14 13:30:47.226358969 (330h42m57.747182714s) [upload=316880]: Uploading segments... May 14 13:30:47.226636899 (330h42m57.747460644s) [upload=316880]: Got next segment. Inline: false May 14 13:30:47.226697074 (330h42m57.747520869s) [upload=316880]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 14 13:30:47.226703506 (330h42m57.747527301s) [upload=316880]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 14 13:30:47.465525228 (330h42m57.986349044s) [seg_pos={0 0}, upload=316880]: Begin upload segment... May 14 13:30:47.465590833 (330h42m57.986414578s) [seg_pos={0 0}, upload=316880]: Done begin upload segment. May 14 13:30:47.465637453 (330h42m57.986474753s) [noise=true , address=193.149.164.69:4086 , piece_id=G6F4G6RYFL2LKLUU5BK4C6YFVJFTGDN46QGYTCR6ZJILB2QAKBQQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.466404867 (330h42m57.987228602s) [noise=true , address=145.79.227.52:4469 , piece_id=HOOUCU52BETFERFHQHRGADFS5CXNPSKDBRUYOR5DHV6YIYZSWZFQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.466633984 (330h42m57.987457729s) [noise=true , address=193.149.164.79:4535 , piece_id=MUMSFNL4A7UURMP4S52E6QDKEMZPWQCBMIARVU7REW7ZPJIGYRPA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467049557 (330h42m57.987873292s) [noise=true , address=77.90.31.14:4685 , piece_id=JKRHMFHPAFDG5DYSOMZDMMUTMKEFMYAZAQ2FSVANPR7ZEAPOP5RA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467363767 (330h42m57.988187512s) [noise=true , address=193.149.164.78:4472 , piece_id=3ZMQGKCDOWN6KF7NDG4Q2WGQZN64HKZLBDFQXAEAS7FNCEYJE4HQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467398434 (330h42m57.988222179s) [noise=true , address=145.79.227.8:4340 , piece_id=I735YQSV2BFCRM3IN4YAOPUMXQHPFNFVIF3OZMWZO47JD55P3EIA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.467667005 (330h42m57.98849075s) [noise=true , address=167.17.72.71:4802 , piece_id=EG67URQ6BSLB72EX6VXDNN4KWMF6JF4JMTMQDCKZODIJENM2H4EA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.468006253 (330h42m57.988829988s) [noise=true , address=193.149.164.70:4128 , piece_id=YQL4CPV6AP3GXDQXPG3PUNJYBSTD7A3KMZDGTJBAJTVBTE777YUQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.468130280 (330h42m57.988954035s) [noise=true , address=45.91.116.14:4203 , piece_id=O7QHYQI4KAK7VAWXH4BALEXGH5JRVWDFGRFJ72MN3FDJ5UVQZIVQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.468822761 (330h42m57.989646506s) [noise=true , address=145.79.227.44:4215 , piece_id=6SWT5RS6AVXU3YQ6YM4OFSEUFMYCJHVU75GEFAF4BT5QHSJ424YQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.469344106 (330h42m57.990167861s) [noise=true , address=193.149.164.71:4300 , piece_id=XUV6Q44FFLERGHWPKH3LTZJXLTB7K5TVJTBXU34LSWPDYCGEVAIQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.469588653 (330h42m57.990412398s) [noise=true , address=193.149.164.67:4030 , piece_id=XZOKQ2HW7XG2RKWPL3C7RJS2LJAC6XJHT7XLOJSQ74MEBDI2FCMQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.470155565 (330h42m57.99097931s) [noise=true , address=167.17.72.70:4772 , piece_id=6BE73P7NXYUYTLIXTAYRAY3AL5YXWAH6US325MLNE5HWLQY2DXGA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.471342059 (330h42m57.992165804s) [noise=true , address=145.79.227.30:4782 , piece_id=ZHAOTMWKAPLBQ7V2XNGVGRXW25A4GKOBZ2SBI4ZFKANJ7VE3IOJQ, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.471793761 (330h42m57.992617506s) [noise=true , address=167.17.72.69:4740 , piece_id=UYGNMGOUAOHJQGAUSUJF2SKB23Z2BXOL24DTIICWFAF46EJKULRA, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:47.472818255 (330h42m57.99364201s) [noise=true , address=45.91.116.10:4086 , piece_id=5NP3LBLYO4KULELOHPO6KWSZPZUW5WWFD6GK64PHR6L2QJ7TEG4Q, seg_pos={0 0}, upload=316880]: Uploading piece... May 14 13:30:48.824555337 (330h42m59.345379072s) [upload=316880]: Next returned nil segment May 14 13:30:48.824567069 (330h42m59.345390814s) [upload=316880]: Waiting for error group managing segments... May 14 13:30:49.208040236 (330h42m59.728863981s) [noise=true , address=167.17.72.71:4802 , piece_id=EG67URQ6BSLB72EX6VXDNN4KWMF6JF4JMTMQDCKZODIJENM2H4EA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.354760988 (330h42m59.875584734s) [noise=true , address=145.79.227.30:4782 , piece_id=ZHAOTMWKAPLBQ7V2XNGVGRXW25A4GKOBZ2SBI4ZFKANJ7VE3IOJQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.355223522 (330h42m59.876047257s) [noise=true , address=145.79.227.8:4340 , piece_id=I735YQSV2BFCRM3IN4YAOPUMXQHPFNFVIF3OZMWZO47JD55P3EIA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.430937189 (330h42m59.951760934s) [noise=true , address=145.79.227.52:4469 , piece_id=HOOUCU52BETFERFHQHRGADFS5CXNPSKDBRUYOR5DHV6YIYZSWZFQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.455996030 (330h42m59.976819775s) [noise=true , address=77.90.31.14:4685 , piece_id=JKRHMFHPAFDG5DYSOMZDMMUTMKEFMYAZAQ2FSVANPR7ZEAPOP5RA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.954832519 (330h43m0.475656264s) [noise=true , address=145.79.227.44:4215 , piece_id=6SWT5RS6AVXU3YQ6YM4OFSEUFMYCJHVU75GEFAF4BT5QHSJ424YQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:49.989917582 (330h43m0.510741327s) [noise=true , address=45.91.116.10:4086 , piece_id=5NP3LBLYO4KULELOHPO6KWSZPZUW5WWFD6GK64PHR6L2QJ7TEG4Q, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:50.082006168 (330h43m0.602829913s) [noise=true , address=45.91.116.14:4203 , piece_id=O7QHYQI4KAK7VAWXH4BALEXGH5JRVWDFGRFJ72MN3FDJ5UVQZIVQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:53.596731701 (330h43m4.117555446s) [noise=true , address=193.149.164.79:4535 , piece_id=MUMSFNL4A7UURMP4S52E6QDKEMZPWQCBMIARVU7REW7ZPJIGYRPA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:55.497462951 (330h43m6.018286696s) [noise=true , address=167.17.72.70:4772 , piece_id=6BE73P7NXYUYTLIXTAYRAY3AL5YXWAH6US325MLNE5HWLQY2DXGA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:56.403300485 (330h43m6.92412423s) [noise=true , address=167.17.72.69:4740 , piece_id=UYGNMGOUAOHJQGAUSUJF2SKB23Z2BXOL24DTIICWFAF46EJKULRA, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:30:59.324401701 (330h43m9.845225446s) [noise=true , address=193.149.164.78:4472 , piece_id=3ZMQGKCDOWN6KF7NDG4Q2WGQZN64HKZLBDFQXAEAS7FNCEYJE4HQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: <nil> May 14 13:31:46.566908056 (330h43m57.087731801s) [noise=true , address=193.149.164.70:4128 , piece_id=YQL4CPV6AP3GXDQXPG3PUNJYBSTD7A3KMZDGTJBAJTVBTE777YUQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:1286axL8SEBRohFVBHVcMGmAc4ktdJQg37LxrknzonqtXWFXiLF, address:193.149.164.70:4128): protocol: write tcp 188.166.225.170:54916->193.149.164.70:4128: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:31:46.566940518 (330h43m57.087764253s) [noise=true , address=193.149.164.67:4030 , piece_id=XZOKQ2HW7XG2RKWPL3C7RJS2LJAC6XJHT7XLOJSQ74MEBDI2FCMQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:1Xs7CkcBfkunXRiLxfEU3KWt497dpskeJJoJTeAeXjesgUN7s8, address:193.149.164.67:4030): protocol: rpc: dial tcp 193.149.164.67:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:31:46.567027874 (330h43m57.087851619s) [noise=true , address=193.149.164.69:4086 , piece_id=G6F4G6RYFL2LKLUU5BK4C6YFVJFTGDN46QGYTCR6ZJILB2QAKBQQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:12kBkPaBtFwEHVi9Ly4WQNGoWMFYa7dHujcJ1GKdqQ4t8vzR9w5, address:193.149.164.69:4086): protocol: write tcp 188.166.225.170:38382->193.149.164.69:4086: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op May 14 13:31:46.567148054 (330h43m57.087971799s) [noise=true , address=193.149.164.71:4300 , piece_id=XUV6Q44FFLERGHWPKH3LTZJXLTB7K5TVJTBXU34LSWPDYCGEVAIQ, seg_pos={0 0}, upload=316880]: Done uploading piece. err: ecclient: upload failed (node:12GRF8go19rLx7rM73XqiV1bBvXaYz3kf8rgfFFA3shmYnkAKZ9, address:193.149.164.71:4300): rpc: dial tcp 193.149.164.71:4300: i/o timeout May 14 13:31:46.567179203 (330h43m57.088002948s) [seg_pos={0 0}, upload=316880]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> May 14 13:31:46.811323383 (330h43m57.332147118s) [upload=316880]: Done uploading segments... May 14 13:31:46.811354222 (330h43m57.332177957s) [upload=316880]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.245983211s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.........................................................XXX] duration: 5.001414778s trace: https://jaeger.dev.storj.io/trace/33e7c8ba4c222528 FAIL (2026-05-14 13:31:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001436299s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........................................................XX] duration: 5.000518305s trace: https://jaeger.dev.storj.io/trace/538658c972daaaed FAIL (2026-05-14 13:31:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000540456s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................................X..................X] duration: 1m0.000748006s trace: https://jaeger.dev.storj.io/trace/17d3d74d64166407 FAIL (2026-05-14 13:31:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000764688s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.000984078s trace: https://jaeger.dev.storj.io/trace/da5d71236db0dd5 FAIL (2026-05-14 13:30:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001003765s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.940039908s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:29:31 UTC) orbiter warning: context deadline exceeded Duration: 6.940068211s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.662603872s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:29:15 UTC) orbiter warning: context deadline exceeded Duration: 6.662627836s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.930743195s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:28:04 UTC) orbiter warning: context deadline exceeded Duration: 6.930763353s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.954357499s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:27:05 UTC) orbiter warning: context deadline exceeded Duration: 6.954380765s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.870802467s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:25:52 UTC) orbiter warning: context deadline exceeded Duration: 6.870830471s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 27.419358738s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:25:17 UTC) orbiter warning: context deadline exceeded Duration: 27.419380609s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.912361992s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:24:11 UTC) orbiter warning: context deadline exceeded Duration: 6.912451324s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 7.003315271s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:22:58 UTC) orbiter warning: context deadline exceeded Duration: 7.003337082s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 7.213419253s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:22:26 UTC) orbiter warning: context deadline exceeded Duration: 7.213444961s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.675681037s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:21:48 UTC) orbiter warning: context deadline exceeded Duration: 6.675709471s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.697474224s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:21:12 UTC) orbiter warning: context deadline exceeded Duration: 6.697503729s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.866196523s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:20:17 UTC) orbiter warning: context deadline exceeded Duration: 6.866221041s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.79759913s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:19:30 UTC) orbiter warning: context deadline exceeded Duration: 6.797623907s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.723911665s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:18:55 UTC) orbiter warning: context deadline exceeded Duration: 6.7239249s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 27.03645726s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:18:48 UTC) orbiter warning: context deadline exceeded Duration: 27.036481277s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.668869186s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:17:47 UTC) orbiter warning: context deadline exceeded Duration: 6.668893922s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.53334085s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:17:05 UTC) orbiter warning: context deadline exceeded Duration: 6.533355828s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.793518115s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:16:12 UTC) orbiter warning: context deadline exceeded Duration: 6.793561318s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.000868906s trace: https://jaeger.dev.storj.io/trace/1a43f3203a73c4c9 FAIL (2026-05-14 13:16:05 UTC) orbiter warning: context deadline exceeded Duration: 1m0.000888443s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [X...........................................................] duration: 3.847347854s trace: https://jaeger.dev.storj.io/trace/445bc53d201009fb FAIL (2026-05-14 12:50:51 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.847371918s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.X..........................................................] duration: 3.871914543s trace: https://jaeger.dev.storj.io/trace/445bc53d201009fb FAIL (2026-05-14 12:50:34 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.871928841s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..X.........................................................] duration: 3.817191425s trace: https://jaeger.dev.storj.io/trace/445bc53d201009fb FAIL (2026-05-14 12:49:27 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.817213006s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...X........................................................] duration: 3.797842794s trace: https://jaeger.dev.storj.io/trace/445bc53d201009fb FAIL (2026-05-14 12:48:55 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.797867972s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....X.......................................................] duration: 3.882559124s trace: https://jaeger.dev.storj.io/trace/445bc53d201009fb FAIL (2026-05-14 12:47:45 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.882581617s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....X......................................................] duration: 3.903403853s trace: https://jaeger.dev.storj.io/trace/445bc53d201009fb FAIL (2026-05-14 12:46:58 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.903418221s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......X.....................................................] duration: 3.888875366s trace: https://jaeger.dev.storj.io/trace/445bc53d201009fb FAIL (2026-05-14 12:46:35 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.88889882s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......X....................................................] duration: 3.953639493s trace: https://jaeger.dev.storj.io/trace/445bc53d201009fb FAIL (2026-05-14 12:45:57 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.953671392s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........X...................................................] duration: 3.806907321s trace: https://jaeger.dev.storj.io/trace/445bc53d201009fb FAIL (2026-05-14 12:44:44 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.80693291s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........X..................................................] duration: 3.877887327s trace: https://jaeger.dev.storj.io/trace/445bc53d201009fb FAIL (2026-05-14 12:44:11 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.877908528s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........X.................................................] duration: 3.891565719s trace: https://jaeger.dev.storj.io/trace/445bc53d201009fb FAIL (2026-05-14 12:43:54 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.891574566s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........X................................................] duration: 3.934756801s trace: https://jaeger.dev.storj.io/trace/445bc53d201009fb FAIL (2026-05-14 12:43:36 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.934779375s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............X...............................................] duration: 4.012789231s trace: https://jaeger.dev.storj.io/trace/445bc53d201009fb FAIL (2026-05-14 12:42:44 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.012807165s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............X..............................................] duration: 3.802557821s trace: https://jaeger.dev.storj.io/trace/445bc53d201009fb FAIL (2026-05-14 12:41:36 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.802572749s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............X.............................................] duration: 3.891690607s trace: https://jaeger.dev.storj.io/trace/445bc53d201009fb FAIL (2026-05-14 12:41:03 UTC) orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.89171316s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)