Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.695327983s trace: https://jaeger.dev.storj.io/trace/6469a32319fb23c3 FAIL (2025-11-02 20:28:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.939448529 (80h17m28.635297282s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940316662 (80h17m28.636165416s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940854095 (80h17m28.636702838s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941163254 (80h17m28.637011997s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941597421 (80h17m28.637446175s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.944379569 (80h17m28.640228322s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945350588 (80h17m28.641199341s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945999965 (80h17m28.641848708s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.946799317 (80h17m28.64264806s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949026458 (80h17m28.644875211s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949648072 (80h17m28.645496815s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:58.374326863 (80h17m30.070175616s) [upload=70665]: Next returned nil segment Nov 2 19:57:58.374346910 (80h17m30.070195674s) [upload=70665]: Waiting for error group managing segments... Nov 2 19:57:58.737256438 (80h17m30.433105181s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.932997019 (80h17m30.628845773s) [noise=true , address=45.91.116.8:4046 , piece_id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.948107422 (80h17m30.643956175s) [noise=true , address=77.90.31.19:4861 , piece_id=DPE7KWFAPMTW67GEMLT2YCBXVMLY75BAU7BLRRYN2UBDIWHUZNJQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.005265203 (80h17m30.701113946s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.052320256 (80h17m30.748169009s) [noise=true , address=77.90.31.13:4658 , piece_id=ZTX6WQ5M3TTA7KUKMJVKWHQHLMRMJW5EUUFVBRBQZDVBFZWWKZQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:58:55.966307021 (80h18m27.662155784s) [noise=true , address=145.79.227.3:4169 , piece_id=GCSPGUWUFDXU2TZ6BNUAEU75A4RUKYCRERUPQRD326RXSDAX7ATQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:145.79.227.3:4169): protocol: rpc: dial tcp 145.79.227.3:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966419525 (80h18m27.662268268s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12jfQbLaQjEXhVTbTPuawAQAbKJSN5RA8kBqz6JVDATQEGAsZ6K, address:145.79.227.13:4833): protocol: rpc: dial tcp 145.79.227.13:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966726459 (80h18m27.662575212s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: rpc: dial tcp 145.79.227.18:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966930668 (80h18m27.662779421s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1F5oTP56RsDAA7Yde8hYvq7Lx5tiFDUHoBvPrYnhQiFy2Rko3N, address:145.79.227.38:4995): protocol: read tcp 188.166.225.170:59394->145.79.227.38:4995: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967260457 (80h18m27.66310921s) [noise=true , address=145.79.227.41:4097 , piece_id=ANQYF4KYYUUGFNUOHEINXGP2VQATR2LNVY6L55FA5FOL5HNAJRGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:145.79.227.41:4097): protocol: rpc: dial tcp 145.79.227.41:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967696617 (80h18m27.66354536s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: rpc: dial tcp 145.79.227.48:4310: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967744338 (80h18m27.663593081s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:145.79.227.8:4357): protocol: rpc: dial tcp 145.79.227.8:4357: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967941153 (80h18m27.663789906s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12HohFM5Npi28Xq65JVMGUDK2bfX7xh5uYXNeHs3XW2VRH8rS72, address:145.79.227.49:4346): protocol: read tcp 188.166.225.170:47702->145.79.227.49:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967952104 (80h18m27.663800847s) [noise=true , address=145.79.227.5:4242 , piece_id=54VY4XFFRB3G5P7VLHFAFUJQTVHVIEHKK2TZT4PNQUOUIFME7MFA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:145.79.227.5:4242): protocol: rpc: dial tcp 145.79.227.5:4242: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968005255 (80h18m27.663854008s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:126wqKRVvj3engw69RJSmdgdB91qMyG1Ed5XfUCpu3ZGq3uXSbe, address:145.79.227.37:4587): protocol: rpc: dial tcp 145.79.227.37:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968119682 (80h18m27.663968426s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: read tcp 188.166.225.170:54484->145.79.227.46:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968176942 (80h18m27.664025695s) [noise=true , address=145.79.227.43:4180 , piece_id=62QSVFJI44O5UMXM7SBZXY7SRMH6ZYSDD23OKDX5WXRLQCYNSXTQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: read tcp 188.166.225.170:48566->145.79.227.43:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968252796 (80h18m27.664101549s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1VzewCRbNpJv7kNwp9wruQuoVSaZFjA44qN33Ct7qm1GnfhtDj, address:145.79.227.45:4237): protocol: read tcp 188.166.225.170:60808->145.79.227.45:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968277563 (80h18m27.664126306s) [seg_pos={0 0}, upload=70665]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Nov 2 19:58:56.213414304 (80h18m27.909263047s) [upload=70665]: Done uploading segments... Nov 2 19:58:56.213452137 (80h18m27.90930088s) [upload=70665]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 5.695356397s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.....................XX.....................................] duration: 3.368727372s trace: https://jaeger.dev.storj.io/trace/1a875a21fbc9493b FAIL (2025-11-02 20:28:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.368770804s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.527328539s trace: https://jaeger.dev.storj.io/trace/6469a32319fb23c3 FAIL (2025-11-02 20:27:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.939448529 (80h17m28.635297282s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940316662 (80h17m28.636165416s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940854095 (80h17m28.636702838s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941163254 (80h17m28.637011997s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941597421 (80h17m28.637446175s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.944379569 (80h17m28.640228322s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945350588 (80h17m28.641199341s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945999965 (80h17m28.641848708s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.946799317 (80h17m28.64264806s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949026458 (80h17m28.644875211s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949648072 (80h17m28.645496815s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:58.374326863 (80h17m30.070175616s) [upload=70665]: Next returned nil segment Nov 2 19:57:58.374346910 (80h17m30.070195674s) [upload=70665]: Waiting for error group managing segments... Nov 2 19:57:58.737256438 (80h17m30.433105181s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.932997019 (80h17m30.628845773s) [noise=true , address=45.91.116.8:4046 , piece_id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.948107422 (80h17m30.643956175s) [noise=true , address=77.90.31.19:4861 , piece_id=DPE7KWFAPMTW67GEMLT2YCBXVMLY75BAU7BLRRYN2UBDIWHUZNJQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.005265203 (80h17m30.701113946s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.052320256 (80h17m30.748169009s) [noise=true , address=77.90.31.13:4658 , piece_id=ZTX6WQ5M3TTA7KUKMJVKWHQHLMRMJW5EUUFVBRBQZDVBFZWWKZQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:58:55.966307021 (80h18m27.662155784s) [noise=true , address=145.79.227.3:4169 , piece_id=GCSPGUWUFDXU2TZ6BNUAEU75A4RUKYCRERUPQRD326RXSDAX7ATQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:145.79.227.3:4169): protocol: rpc: dial tcp 145.79.227.3:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966419525 (80h18m27.662268268s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12jfQbLaQjEXhVTbTPuawAQAbKJSN5RA8kBqz6JVDATQEGAsZ6K, address:145.79.227.13:4833): protocol: rpc: dial tcp 145.79.227.13:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966726459 (80h18m27.662575212s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: rpc: dial tcp 145.79.227.18:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966930668 (80h18m27.662779421s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1F5oTP56RsDAA7Yde8hYvq7Lx5tiFDUHoBvPrYnhQiFy2Rko3N, address:145.79.227.38:4995): protocol: read tcp 188.166.225.170:59394->145.79.227.38:4995: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967260457 (80h18m27.66310921s) [noise=true , address=145.79.227.41:4097 , piece_id=ANQYF4KYYUUGFNUOHEINXGP2VQATR2LNVY6L55FA5FOL5HNAJRGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:145.79.227.41:4097): protocol: rpc: dial tcp 145.79.227.41:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967696617 (80h18m27.66354536s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: rpc: dial tcp 145.79.227.48:4310: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967744338 (80h18m27.663593081s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:145.79.227.8:4357): protocol: rpc: dial tcp 145.79.227.8:4357: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967941153 (80h18m27.663789906s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12HohFM5Npi28Xq65JVMGUDK2bfX7xh5uYXNeHs3XW2VRH8rS72, address:145.79.227.49:4346): protocol: read tcp 188.166.225.170:47702->145.79.227.49:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967952104 (80h18m27.663800847s) [noise=true , address=145.79.227.5:4242 , piece_id=54VY4XFFRB3G5P7VLHFAFUJQTVHVIEHKK2TZT4PNQUOUIFME7MFA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:145.79.227.5:4242): protocol: rpc: dial tcp 145.79.227.5:4242: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968005255 (80h18m27.663854008s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:126wqKRVvj3engw69RJSmdgdB91qMyG1Ed5XfUCpu3ZGq3uXSbe, address:145.79.227.37:4587): protocol: rpc: dial tcp 145.79.227.37:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968119682 (80h18m27.663968426s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: read tcp 188.166.225.170:54484->145.79.227.46:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968176942 (80h18m27.664025695s) [noise=true , address=145.79.227.43:4180 , piece_id=62QSVFJI44O5UMXM7SBZXY7SRMH6ZYSDD23OKDX5WXRLQCYNSXTQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: read tcp 188.166.225.170:48566->145.79.227.43:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968252796 (80h18m27.664101549s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1VzewCRbNpJv7kNwp9wruQuoVSaZFjA44qN33Ct7qm1GnfhtDj, address:145.79.227.45:4237): protocol: read tcp 188.166.225.170:60808->145.79.227.45:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968277563 (80h18m27.664126306s) [seg_pos={0 0}, upload=70665]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Nov 2 19:58:56.213414304 (80h18m27.909263047s) [upload=70665]: Done uploading segments... Nov 2 19:58:56.213452137 (80h18m27.90930088s) [upload=70665]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 5.527357555s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [......................XX....................................] duration: 3.408695149s trace: https://jaeger.dev.storj.io/trace/1a875a21fbc9493b FAIL (2025-11-02 20:27:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.408740666s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.......................XX...................................] duration: 3.420922637s trace: https://jaeger.dev.storj.io/trace/1a875a21fbc9493b FAIL (2025-11-02 20:27:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.420974184s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.674350258s trace: https://jaeger.dev.storj.io/trace/6469a32319fb23c3 FAIL (2025-11-02 20:27:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.939448529 (80h17m28.635297282s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940316662 (80h17m28.636165416s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940854095 (80h17m28.636702838s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941163254 (80h17m28.637011997s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941597421 (80h17m28.637446175s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.944379569 (80h17m28.640228322s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945350588 (80h17m28.641199341s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945999965 (80h17m28.641848708s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.946799317 (80h17m28.64264806s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949026458 (80h17m28.644875211s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949648072 (80h17m28.645496815s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:58.374326863 (80h17m30.070175616s) [upload=70665]: Next returned nil segment Nov 2 19:57:58.374346910 (80h17m30.070195674s) [upload=70665]: Waiting for error group managing segments... Nov 2 19:57:58.737256438 (80h17m30.433105181s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.932997019 (80h17m30.628845773s) [noise=true , address=45.91.116.8:4046 , piece_id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.948107422 (80h17m30.643956175s) [noise=true , address=77.90.31.19:4861 , piece_id=DPE7KWFAPMTW67GEMLT2YCBXVMLY75BAU7BLRRYN2UBDIWHUZNJQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.005265203 (80h17m30.701113946s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.052320256 (80h17m30.748169009s) [noise=true , address=77.90.31.13:4658 , piece_id=ZTX6WQ5M3TTA7KUKMJVKWHQHLMRMJW5EUUFVBRBQZDVBFZWWKZQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:58:55.966307021 (80h18m27.662155784s) [noise=true , address=145.79.227.3:4169 , piece_id=GCSPGUWUFDXU2TZ6BNUAEU75A4RUKYCRERUPQRD326RXSDAX7ATQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:145.79.227.3:4169): protocol: rpc: dial tcp 145.79.227.3:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966419525 (80h18m27.662268268s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12jfQbLaQjEXhVTbTPuawAQAbKJSN5RA8kBqz6JVDATQEGAsZ6K, address:145.79.227.13:4833): protocol: rpc: dial tcp 145.79.227.13:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966726459 (80h18m27.662575212s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: rpc: dial tcp 145.79.227.18:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966930668 (80h18m27.662779421s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1F5oTP56RsDAA7Yde8hYvq7Lx5tiFDUHoBvPrYnhQiFy2Rko3N, address:145.79.227.38:4995): protocol: read tcp 188.166.225.170:59394->145.79.227.38:4995: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967260457 (80h18m27.66310921s) [noise=true , address=145.79.227.41:4097 , piece_id=ANQYF4KYYUUGFNUOHEINXGP2VQATR2LNVY6L55FA5FOL5HNAJRGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:145.79.227.41:4097): protocol: rpc: dial tcp 145.79.227.41:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967696617 (80h18m27.66354536s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: rpc: dial tcp 145.79.227.48:4310: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967744338 (80h18m27.663593081s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:145.79.227.8:4357): protocol: rpc: dial tcp 145.79.227.8:4357: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967941153 (80h18m27.663789906s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12HohFM5Npi28Xq65JVMGUDK2bfX7xh5uYXNeHs3XW2VRH8rS72, address:145.79.227.49:4346): protocol: read tcp 188.166.225.170:47702->145.79.227.49:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967952104 (80h18m27.663800847s) [noise=true , address=145.79.227.5:4242 , piece_id=54VY4XFFRB3G5P7VLHFAFUJQTVHVIEHKK2TZT4PNQUOUIFME7MFA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:145.79.227.5:4242): protocol: rpc: dial tcp 145.79.227.5:4242: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968005255 (80h18m27.663854008s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:126wqKRVvj3engw69RJSmdgdB91qMyG1Ed5XfUCpu3ZGq3uXSbe, address:145.79.227.37:4587): protocol: rpc: dial tcp 145.79.227.37:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968119682 (80h18m27.663968426s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: read tcp 188.166.225.170:54484->145.79.227.46:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968176942 (80h18m27.664025695s) [noise=true , address=145.79.227.43:4180 , piece_id=62QSVFJI44O5UMXM7SBZXY7SRMH6ZYSDD23OKDX5WXRLQCYNSXTQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: read tcp 188.166.225.170:48566->145.79.227.43:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968252796 (80h18m27.664101549s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1VzewCRbNpJv7kNwp9wruQuoVSaZFjA44qN33Ct7qm1GnfhtDj, address:145.79.227.45:4237): protocol: read tcp 188.166.225.170:60808->145.79.227.45:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968277563 (80h18m27.664126306s) [seg_pos={0 0}, upload=70665]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Nov 2 19:58:56.213414304 (80h18m27.909263047s) [upload=70665]: Done uploading segments... Nov 2 19:58:56.213452137 (80h18m27.90930088s) [upload=70665]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 5.674380566s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 313.868913ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/42eaea040e9ca2bf FAIL (2025-11-02 20:26:38 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42676->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 313.904802ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.622496748s trace: https://jaeger.dev.storj.io/trace/6469a32319fb23c3 FAIL (2025-11-02 20:26:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.939448529 (80h17m28.635297282s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940316662 (80h17m28.636165416s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940854095 (80h17m28.636702838s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941163254 (80h17m28.637011997s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941597421 (80h17m28.637446175s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.944379569 (80h17m28.640228322s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945350588 (80h17m28.641199341s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945999965 (80h17m28.641848708s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.946799317 (80h17m28.64264806s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949026458 (80h17m28.644875211s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949648072 (80h17m28.645496815s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:58.374326863 (80h17m30.070175616s) [upload=70665]: Next returned nil segment Nov 2 19:57:58.374346910 (80h17m30.070195674s) [upload=70665]: Waiting for error group managing segments... Nov 2 19:57:58.737256438 (80h17m30.433105181s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.932997019 (80h17m30.628845773s) [noise=true , address=45.91.116.8:4046 , piece_id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.948107422 (80h17m30.643956175s) [noise=true , address=77.90.31.19:4861 , piece_id=DPE7KWFAPMTW67GEMLT2YCBXVMLY75BAU7BLRRYN2UBDIWHUZNJQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.005265203 (80h17m30.701113946s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.052320256 (80h17m30.748169009s) [noise=true , address=77.90.31.13:4658 , piece_id=ZTX6WQ5M3TTA7KUKMJVKWHQHLMRMJW5EUUFVBRBQZDVBFZWWKZQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:58:55.966307021 (80h18m27.662155784s) [noise=true , address=145.79.227.3:4169 , piece_id=GCSPGUWUFDXU2TZ6BNUAEU75A4RUKYCRERUPQRD326RXSDAX7ATQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:145.79.227.3:4169): protocol: rpc: dial tcp 145.79.227.3:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966419525 (80h18m27.662268268s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12jfQbLaQjEXhVTbTPuawAQAbKJSN5RA8kBqz6JVDATQEGAsZ6K, address:145.79.227.13:4833): protocol: rpc: dial tcp 145.79.227.13:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966726459 (80h18m27.662575212s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: rpc: dial tcp 145.79.227.18:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966930668 (80h18m27.662779421s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1F5oTP56RsDAA7Yde8hYvq7Lx5tiFDUHoBvPrYnhQiFy2Rko3N, address:145.79.227.38:4995): protocol: read tcp 188.166.225.170:59394->145.79.227.38:4995: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967260457 (80h18m27.66310921s) [noise=true , address=145.79.227.41:4097 , piece_id=ANQYF4KYYUUGFNUOHEINXGP2VQATR2LNVY6L55FA5FOL5HNAJRGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:145.79.227.41:4097): protocol: rpc: dial tcp 145.79.227.41:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967696617 (80h18m27.66354536s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: rpc: dial tcp 145.79.227.48:4310: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967744338 (80h18m27.663593081s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:145.79.227.8:4357): protocol: rpc: dial tcp 145.79.227.8:4357: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967941153 (80h18m27.663789906s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12HohFM5Npi28Xq65JVMGUDK2bfX7xh5uYXNeHs3XW2VRH8rS72, address:145.79.227.49:4346): protocol: read tcp 188.166.225.170:47702->145.79.227.49:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967952104 (80h18m27.663800847s) [noise=true , address=145.79.227.5:4242 , piece_id=54VY4XFFRB3G5P7VLHFAFUJQTVHVIEHKK2TZT4PNQUOUIFME7MFA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:145.79.227.5:4242): protocol: rpc: dial tcp 145.79.227.5:4242: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968005255 (80h18m27.663854008s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:126wqKRVvj3engw69RJSmdgdB91qMyG1Ed5XfUCpu3ZGq3uXSbe, address:145.79.227.37:4587): protocol: rpc: dial tcp 145.79.227.37:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968119682 (80h18m27.663968426s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: read tcp 188.166.225.170:54484->145.79.227.46:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968176942 (80h18m27.664025695s) [noise=true , address=145.79.227.43:4180 , piece_id=62QSVFJI44O5UMXM7SBZXY7SRMH6ZYSDD23OKDX5WXRLQCYNSXTQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: read tcp 188.166.225.170:48566->145.79.227.43:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968252796 (80h18m27.664101549s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1VzewCRbNpJv7kNwp9wruQuoVSaZFjA44qN33Ct7qm1GnfhtDj, address:145.79.227.45:4237): protocol: read tcp 188.166.225.170:60808->145.79.227.45:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968277563 (80h18m27.664126306s) [seg_pos={0 0}, upload=70665]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Nov 2 19:58:56.213414304 (80h18m27.909263047s) [upload=70665]: Done uploading segments... Nov 2 19:58:56.213452137 (80h18m27.90930088s) [upload=70665]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 5.622525191s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [........................XX..................................] duration: 3.440577721s trace: https://jaeger.dev.storj.io/trace/1a875a21fbc9493b FAIL (2025-11-02 20:26:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.440594213s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 319.871241ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/42eaea040e9ca2bf FAIL (2025-11-02 20:25:47 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42676->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 319.902091ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.632957972s trace: https://jaeger.dev.storj.io/trace/6469a32319fb23c3 FAIL (2025-11-02 20:25:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.939448529 (80h17m28.635297282s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940316662 (80h17m28.636165416s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940854095 (80h17m28.636702838s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941163254 (80h17m28.637011997s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941597421 (80h17m28.637446175s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.944379569 (80h17m28.640228322s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945350588 (80h17m28.641199341s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945999965 (80h17m28.641848708s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.946799317 (80h17m28.64264806s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949026458 (80h17m28.644875211s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949648072 (80h17m28.645496815s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:58.374326863 (80h17m30.070175616s) [upload=70665]: Next returned nil segment Nov 2 19:57:58.374346910 (80h17m30.070195674s) [upload=70665]: Waiting for error group managing segments... Nov 2 19:57:58.737256438 (80h17m30.433105181s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.932997019 (80h17m30.628845773s) [noise=true , address=45.91.116.8:4046 , piece_id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.948107422 (80h17m30.643956175s) [noise=true , address=77.90.31.19:4861 , piece_id=DPE7KWFAPMTW67GEMLT2YCBXVMLY75BAU7BLRRYN2UBDIWHUZNJQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.005265203 (80h17m30.701113946s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.052320256 (80h17m30.748169009s) [noise=true , address=77.90.31.13:4658 , piece_id=ZTX6WQ5M3TTA7KUKMJVKWHQHLMRMJW5EUUFVBRBQZDVBFZWWKZQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:58:55.966307021 (80h18m27.662155784s) [noise=true , address=145.79.227.3:4169 , piece_id=GCSPGUWUFDXU2TZ6BNUAEU75A4RUKYCRERUPQRD326RXSDAX7ATQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:145.79.227.3:4169): protocol: rpc: dial tcp 145.79.227.3:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966419525 (80h18m27.662268268s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12jfQbLaQjEXhVTbTPuawAQAbKJSN5RA8kBqz6JVDATQEGAsZ6K, address:145.79.227.13:4833): protocol: rpc: dial tcp 145.79.227.13:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966726459 (80h18m27.662575212s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: rpc: dial tcp 145.79.227.18:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966930668 (80h18m27.662779421s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1F5oTP56RsDAA7Yde8hYvq7Lx5tiFDUHoBvPrYnhQiFy2Rko3N, address:145.79.227.38:4995): protocol: read tcp 188.166.225.170:59394->145.79.227.38:4995: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967260457 (80h18m27.66310921s) [noise=true , address=145.79.227.41:4097 , piece_id=ANQYF4KYYUUGFNUOHEINXGP2VQATR2LNVY6L55FA5FOL5HNAJRGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:145.79.227.41:4097): protocol: rpc: dial tcp 145.79.227.41:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967696617 (80h18m27.66354536s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: rpc: dial tcp 145.79.227.48:4310: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967744338 (80h18m27.663593081s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:145.79.227.8:4357): protocol: rpc: dial tcp 145.79.227.8:4357: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967941153 (80h18m27.663789906s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12HohFM5Npi28Xq65JVMGUDK2bfX7xh5uYXNeHs3XW2VRH8rS72, address:145.79.227.49:4346): protocol: read tcp 188.166.225.170:47702->145.79.227.49:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967952104 (80h18m27.663800847s) [noise=true , address=145.79.227.5:4242 , piece_id=54VY4XFFRB3G5P7VLHFAFUJQTVHVIEHKK2TZT4PNQUOUIFME7MFA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:145.79.227.5:4242): protocol: rpc: dial tcp 145.79.227.5:4242: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968005255 (80h18m27.663854008s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:126wqKRVvj3engw69RJSmdgdB91qMyG1Ed5XfUCpu3ZGq3uXSbe, address:145.79.227.37:4587): protocol: rpc: dial tcp 145.79.227.37:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968119682 (80h18m27.663968426s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: read tcp 188.166.225.170:54484->145.79.227.46:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968176942 (80h18m27.664025695s) [noise=true , address=145.79.227.43:4180 , piece_id=62QSVFJI44O5UMXM7SBZXY7SRMH6ZYSDD23OKDX5WXRLQCYNSXTQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: read tcp 188.166.225.170:48566->145.79.227.43:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968252796 (80h18m27.664101549s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1VzewCRbNpJv7kNwp9wruQuoVSaZFjA44qN33Ct7qm1GnfhtDj, address:145.79.227.45:4237): protocol: read tcp 188.166.225.170:60808->145.79.227.45:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968277563 (80h18m27.664126306s) [seg_pos={0 0}, upload=70665]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Nov 2 19:58:56.213414304 (80h18m27.909263047s) [upload=70665]: Done uploading segments... Nov 2 19:58:56.213452137 (80h18m27.90930088s) [upload=70665]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 5.632995674s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.........................XX.................................] duration: 3.446735977s trace: https://jaeger.dev.storj.io/trace/1a875a21fbc9493b FAIL (2025-11-02 20:25:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.446787826s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 346.51624ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/580c04fa88dfbdac FAIL (2025-11-02 20:25:08 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:42672->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 346.544634ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 319.741455ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/42eaea040e9ca2bf FAIL (2025-11-02 20:24:34 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42676->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 319.766904ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.556827444s trace: https://jaeger.dev.storj.io/trace/6469a32319fb23c3 FAIL (2025-11-02 20:24:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.939448529 (80h17m28.635297282s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940316662 (80h17m28.636165416s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940854095 (80h17m28.636702838s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941163254 (80h17m28.637011997s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941597421 (80h17m28.637446175s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.944379569 (80h17m28.640228322s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945350588 (80h17m28.641199341s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945999965 (80h17m28.641848708s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.946799317 (80h17m28.64264806s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949026458 (80h17m28.644875211s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949648072 (80h17m28.645496815s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:58.374326863 (80h17m30.070175616s) [upload=70665]: Next returned nil segment Nov 2 19:57:58.374346910 (80h17m30.070195674s) [upload=70665]: Waiting for error group managing segments... Nov 2 19:57:58.737256438 (80h17m30.433105181s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.932997019 (80h17m30.628845773s) [noise=true , address=45.91.116.8:4046 , piece_id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.948107422 (80h17m30.643956175s) [noise=true , address=77.90.31.19:4861 , piece_id=DPE7KWFAPMTW67GEMLT2YCBXVMLY75BAU7BLRRYN2UBDIWHUZNJQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.005265203 (80h17m30.701113946s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.052320256 (80h17m30.748169009s) [noise=true , address=77.90.31.13:4658 , piece_id=ZTX6WQ5M3TTA7KUKMJVKWHQHLMRMJW5EUUFVBRBQZDVBFZWWKZQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:58:55.966307021 (80h18m27.662155784s) [noise=true , address=145.79.227.3:4169 , piece_id=GCSPGUWUFDXU2TZ6BNUAEU75A4RUKYCRERUPQRD326RXSDAX7ATQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:145.79.227.3:4169): protocol: rpc: dial tcp 145.79.227.3:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966419525 (80h18m27.662268268s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12jfQbLaQjEXhVTbTPuawAQAbKJSN5RA8kBqz6JVDATQEGAsZ6K, address:145.79.227.13:4833): protocol: rpc: dial tcp 145.79.227.13:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966726459 (80h18m27.662575212s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: rpc: dial tcp 145.79.227.18:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966930668 (80h18m27.662779421s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1F5oTP56RsDAA7Yde8hYvq7Lx5tiFDUHoBvPrYnhQiFy2Rko3N, address:145.79.227.38:4995): protocol: read tcp 188.166.225.170:59394->145.79.227.38:4995: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967260457 (80h18m27.66310921s) [noise=true , address=145.79.227.41:4097 , piece_id=ANQYF4KYYUUGFNUOHEINXGP2VQATR2LNVY6L55FA5FOL5HNAJRGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:145.79.227.41:4097): protocol: rpc: dial tcp 145.79.227.41:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967696617 (80h18m27.66354536s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: rpc: dial tcp 145.79.227.48:4310: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967744338 (80h18m27.663593081s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:145.79.227.8:4357): protocol: rpc: dial tcp 145.79.227.8:4357: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967941153 (80h18m27.663789906s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12HohFM5Npi28Xq65JVMGUDK2bfX7xh5uYXNeHs3XW2VRH8rS72, address:145.79.227.49:4346): protocol: read tcp 188.166.225.170:47702->145.79.227.49:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967952104 (80h18m27.663800847s) [noise=true , address=145.79.227.5:4242 , piece_id=54VY4XFFRB3G5P7VLHFAFUJQTVHVIEHKK2TZT4PNQUOUIFME7MFA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:145.79.227.5:4242): protocol: rpc: dial tcp 145.79.227.5:4242: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968005255 (80h18m27.663854008s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:126wqKRVvj3engw69RJSmdgdB91qMyG1Ed5XfUCpu3ZGq3uXSbe, address:145.79.227.37:4587): protocol: rpc: dial tcp 145.79.227.37:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968119682 (80h18m27.663968426s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: read tcp 188.166.225.170:54484->145.79.227.46:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968176942 (80h18m27.664025695s) [noise=true , address=145.79.227.43:4180 , piece_id=62QSVFJI44O5UMXM7SBZXY7SRMH6ZYSDD23OKDX5WXRLQCYNSXTQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: read tcp 188.166.225.170:48566->145.79.227.43:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968252796 (80h18m27.664101549s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1VzewCRbNpJv7kNwp9wruQuoVSaZFjA44qN33Ct7qm1GnfhtDj, address:145.79.227.45:4237): protocol: read tcp 188.166.225.170:60808->145.79.227.45:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968277563 (80h18m27.664126306s) [seg_pos={0 0}, upload=70665]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Nov 2 19:58:56.213414304 (80h18m27.909263047s) [upload=70665]: Done uploading segments... Nov 2 19:58:56.213452137 (80h18m27.90930088s) [upload=70665]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 5.556855897s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........................XX................................] duration: 3.427859264s trace: https://jaeger.dev.storj.io/trace/1a875a21fbc9493b FAIL (2025-11-02 20:24:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.427891746s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 44.08669ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/580c04fa88dfbdac FAIL (2025-11-02 20:23:57 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:42672->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 44.114583ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 324.996172ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/42eaea040e9ca2bf FAIL (2025-11-02 20:23:51 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42676->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 325.025899ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...........................XX...............................] duration: 3.409349144s trace: https://jaeger.dev.storj.io/trace/1a875a21fbc9493b FAIL (2025-11-02 20:23:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.409437433s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.777199406s trace: https://jaeger.dev.storj.io/trace/6469a32319fb23c3 FAIL (2025-11-02 20:23:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.939448529 (80h17m28.635297282s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940316662 (80h17m28.636165416s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940854095 (80h17m28.636702838s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941163254 (80h17m28.637011997s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941597421 (80h17m28.637446175s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.944379569 (80h17m28.640228322s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945350588 (80h17m28.641199341s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945999965 (80h17m28.641848708s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.946799317 (80h17m28.64264806s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949026458 (80h17m28.644875211s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949648072 (80h17m28.645496815s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:58.374326863 (80h17m30.070175616s) [upload=70665]: Next returned nil segment Nov 2 19:57:58.374346910 (80h17m30.070195674s) [upload=70665]: Waiting for error group managing segments... Nov 2 19:57:58.737256438 (80h17m30.433105181s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.932997019 (80h17m30.628845773s) [noise=true , address=45.91.116.8:4046 , piece_id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.948107422 (80h17m30.643956175s) [noise=true , address=77.90.31.19:4861 , piece_id=DPE7KWFAPMTW67GEMLT2YCBXVMLY75BAU7BLRRYN2UBDIWHUZNJQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.005265203 (80h17m30.701113946s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.052320256 (80h17m30.748169009s) [noise=true , address=77.90.31.13:4658 , piece_id=ZTX6WQ5M3TTA7KUKMJVKWHQHLMRMJW5EUUFVBRBQZDVBFZWWKZQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:58:55.966307021 (80h18m27.662155784s) [noise=true , address=145.79.227.3:4169 , piece_id=GCSPGUWUFDXU2TZ6BNUAEU75A4RUKYCRERUPQRD326RXSDAX7ATQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:145.79.227.3:4169): protocol: rpc: dial tcp 145.79.227.3:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966419525 (80h18m27.662268268s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12jfQbLaQjEXhVTbTPuawAQAbKJSN5RA8kBqz6JVDATQEGAsZ6K, address:145.79.227.13:4833): protocol: rpc: dial tcp 145.79.227.13:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966726459 (80h18m27.662575212s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: rpc: dial tcp 145.79.227.18:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966930668 (80h18m27.662779421s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1F5oTP56RsDAA7Yde8hYvq7Lx5tiFDUHoBvPrYnhQiFy2Rko3N, address:145.79.227.38:4995): protocol: read tcp 188.166.225.170:59394->145.79.227.38:4995: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967260457 (80h18m27.66310921s) [noise=true , address=145.79.227.41:4097 , piece_id=ANQYF4KYYUUGFNUOHEINXGP2VQATR2LNVY6L55FA5FOL5HNAJRGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:145.79.227.41:4097): protocol: rpc: dial tcp 145.79.227.41:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967696617 (80h18m27.66354536s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: rpc: dial tcp 145.79.227.48:4310: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967744338 (80h18m27.663593081s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:145.79.227.8:4357): protocol: rpc: dial tcp 145.79.227.8:4357: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967941153 (80h18m27.663789906s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12HohFM5Npi28Xq65JVMGUDK2bfX7xh5uYXNeHs3XW2VRH8rS72, address:145.79.227.49:4346): protocol: read tcp 188.166.225.170:47702->145.79.227.49:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967952104 (80h18m27.663800847s) [noise=true , address=145.79.227.5:4242 , piece_id=54VY4XFFRB3G5P7VLHFAFUJQTVHVIEHKK2TZT4PNQUOUIFME7MFA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:145.79.227.5:4242): protocol: rpc: dial tcp 145.79.227.5:4242: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968005255 (80h18m27.663854008s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:126wqKRVvj3engw69RJSmdgdB91qMyG1Ed5XfUCpu3ZGq3uXSbe, address:145.79.227.37:4587): protocol: rpc: dial tcp 145.79.227.37:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968119682 (80h18m27.663968426s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: read tcp 188.166.225.170:54484->145.79.227.46:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968176942 (80h18m27.664025695s) [noise=true , address=145.79.227.43:4180 , piece_id=62QSVFJI44O5UMXM7SBZXY7SRMH6ZYSDD23OKDX5WXRLQCYNSXTQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: read tcp 188.166.225.170:48566->145.79.227.43:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968252796 (80h18m27.664101549s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1VzewCRbNpJv7kNwp9wruQuoVSaZFjA44qN33Ct7qm1GnfhtDj, address:145.79.227.45:4237): protocol: read tcp 188.166.225.170:60808->145.79.227.45:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968277563 (80h18m27.664126306s) [seg_pos={0 0}, upload=70665]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Nov 2 19:58:56.213414304 (80h18m27.909263047s) [upload=70665]: Done uploading segments... Nov 2 19:58:56.213452137 (80h18m27.90930088s) [upload=70665]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 5.777226369s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 52.600903ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/580c04fa88dfbdac FAIL (2025-11-02 20:23:20 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:42672->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 52.629759ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 625.106806ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/42eaea040e9ca2bf FAIL (2025-11-02 20:23:03 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42676->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 625.143225ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.680198177s trace: https://jaeger.dev.storj.io/trace/6469a32319fb23c3 FAIL (2025-11-02 20:23:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.939448529 (80h17m28.635297282s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940316662 (80h17m28.636165416s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940854095 (80h17m28.636702838s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941163254 (80h17m28.637011997s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941597421 (80h17m28.637446175s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.944379569 (80h17m28.640228322s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945350588 (80h17m28.641199341s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945999965 (80h17m28.641848708s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.946799317 (80h17m28.64264806s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949026458 (80h17m28.644875211s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949648072 (80h17m28.645496815s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:58.374326863 (80h17m30.070175616s) [upload=70665]: Next returned nil segment Nov 2 19:57:58.374346910 (80h17m30.070195674s) [upload=70665]: Waiting for error group managing segments... Nov 2 19:57:58.737256438 (80h17m30.433105181s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.932997019 (80h17m30.628845773s) [noise=true , address=45.91.116.8:4046 , piece_id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.948107422 (80h17m30.643956175s) [noise=true , address=77.90.31.19:4861 , piece_id=DPE7KWFAPMTW67GEMLT2YCBXVMLY75BAU7BLRRYN2UBDIWHUZNJQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.005265203 (80h17m30.701113946s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.052320256 (80h17m30.748169009s) [noise=true , address=77.90.31.13:4658 , piece_id=ZTX6WQ5M3TTA7KUKMJVKWHQHLMRMJW5EUUFVBRBQZDVBFZWWKZQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:58:55.966307021 (80h18m27.662155784s) [noise=true , address=145.79.227.3:4169 , piece_id=GCSPGUWUFDXU2TZ6BNUAEU75A4RUKYCRERUPQRD326RXSDAX7ATQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:145.79.227.3:4169): protocol: rpc: dial tcp 145.79.227.3:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966419525 (80h18m27.662268268s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12jfQbLaQjEXhVTbTPuawAQAbKJSN5RA8kBqz6JVDATQEGAsZ6K, address:145.79.227.13:4833): protocol: rpc: dial tcp 145.79.227.13:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966726459 (80h18m27.662575212s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: rpc: dial tcp 145.79.227.18:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966930668 (80h18m27.662779421s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1F5oTP56RsDAA7Yde8hYvq7Lx5tiFDUHoBvPrYnhQiFy2Rko3N, address:145.79.227.38:4995): protocol: read tcp 188.166.225.170:59394->145.79.227.38:4995: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967260457 (80h18m27.66310921s) [noise=true , address=145.79.227.41:4097 , piece_id=ANQYF4KYYUUGFNUOHEINXGP2VQATR2LNVY6L55FA5FOL5HNAJRGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:145.79.227.41:4097): protocol: rpc: dial tcp 145.79.227.41:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967696617 (80h18m27.66354536s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: rpc: dial tcp 145.79.227.48:4310: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967744338 (80h18m27.663593081s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:145.79.227.8:4357): protocol: rpc: dial tcp 145.79.227.8:4357: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967941153 (80h18m27.663789906s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12HohFM5Npi28Xq65JVMGUDK2bfX7xh5uYXNeHs3XW2VRH8rS72, address:145.79.227.49:4346): protocol: read tcp 188.166.225.170:47702->145.79.227.49:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967952104 (80h18m27.663800847s) [noise=true , address=145.79.227.5:4242 , piece_id=54VY4XFFRB3G5P7VLHFAFUJQTVHVIEHKK2TZT4PNQUOUIFME7MFA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:145.79.227.5:4242): protocol: rpc: dial tcp 145.79.227.5:4242: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968005255 (80h18m27.663854008s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:126wqKRVvj3engw69RJSmdgdB91qMyG1Ed5XfUCpu3ZGq3uXSbe, address:145.79.227.37:4587): protocol: rpc: dial tcp 145.79.227.37:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968119682 (80h18m27.663968426s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: read tcp 188.166.225.170:54484->145.79.227.46:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968176942 (80h18m27.664025695s) [noise=true , address=145.79.227.43:4180 , piece_id=62QSVFJI44O5UMXM7SBZXY7SRMH6ZYSDD23OKDX5WXRLQCYNSXTQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: read tcp 188.166.225.170:48566->145.79.227.43:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968252796 (80h18m27.664101549s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1VzewCRbNpJv7kNwp9wruQuoVSaZFjA44qN33Ct7qm1GnfhtDj, address:145.79.227.45:4237): protocol: read tcp 188.166.225.170:60808->145.79.227.45:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968277563 (80h18m27.664126306s) [seg_pos={0 0}, upload=70665]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Nov 2 19:58:56.213414304 (80h18m27.909263047s) [upload=70665]: Done uploading segments... Nov 2 19:58:56.213452137 (80h18m27.90930088s) [upload=70665]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 5.680224286s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 47.00634ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/580c04fa88dfbdac FAIL (2025-11-02 20:22:46 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:42672->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 47.036928ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.647907174s trace: https://jaeger.dev.storj.io/trace/6469a32319fb23c3 FAIL (2025-11-02 20:22:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.939448529 (80h17m28.635297282s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940316662 (80h17m28.636165416s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940854095 (80h17m28.636702838s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941163254 (80h17m28.637011997s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941597421 (80h17m28.637446175s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.944379569 (80h17m28.640228322s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945350588 (80h17m28.641199341s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945999965 (80h17m28.641848708s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.946799317 (80h17m28.64264806s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949026458 (80h17m28.644875211s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949648072 (80h17m28.645496815s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:58.374326863 (80h17m30.070175616s) [upload=70665]: Next returned nil segment Nov 2 19:57:58.374346910 (80h17m30.070195674s) [upload=70665]: Waiting for error group managing segments... Nov 2 19:57:58.737256438 (80h17m30.433105181s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.932997019 (80h17m30.628845773s) [noise=true , address=45.91.116.8:4046 , piece_id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.948107422 (80h17m30.643956175s) [noise=true , address=77.90.31.19:4861 , piece_id=DPE7KWFAPMTW67GEMLT2YCBXVMLY75BAU7BLRRYN2UBDIWHUZNJQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.005265203 (80h17m30.701113946s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.052320256 (80h17m30.748169009s) [noise=true , address=77.90.31.13:4658 , piece_id=ZTX6WQ5M3TTA7KUKMJVKWHQHLMRMJW5EUUFVBRBQZDVBFZWWKZQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:58:55.966307021 (80h18m27.662155784s) [noise=true , address=145.79.227.3:4169 , piece_id=GCSPGUWUFDXU2TZ6BNUAEU75A4RUKYCRERUPQRD326RXSDAX7ATQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:145.79.227.3:4169): protocol: rpc: dial tcp 145.79.227.3:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966419525 (80h18m27.662268268s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12jfQbLaQjEXhVTbTPuawAQAbKJSN5RA8kBqz6JVDATQEGAsZ6K, address:145.79.227.13:4833): protocol: rpc: dial tcp 145.79.227.13:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966726459 (80h18m27.662575212s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: rpc: dial tcp 145.79.227.18:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966930668 (80h18m27.662779421s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1F5oTP56RsDAA7Yde8hYvq7Lx5tiFDUHoBvPrYnhQiFy2Rko3N, address:145.79.227.38:4995): protocol: read tcp 188.166.225.170:59394->145.79.227.38:4995: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967260457 (80h18m27.66310921s) [noise=true , address=145.79.227.41:4097 , piece_id=ANQYF4KYYUUGFNUOHEINXGP2VQATR2LNVY6L55FA5FOL5HNAJRGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:145.79.227.41:4097): protocol: rpc: dial tcp 145.79.227.41:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967696617 (80h18m27.66354536s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: rpc: dial tcp 145.79.227.48:4310: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967744338 (80h18m27.663593081s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:145.79.227.8:4357): protocol: rpc: dial tcp 145.79.227.8:4357: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967941153 (80h18m27.663789906s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12HohFM5Npi28Xq65JVMGUDK2bfX7xh5uYXNeHs3XW2VRH8rS72, address:145.79.227.49:4346): protocol: read tcp 188.166.225.170:47702->145.79.227.49:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967952104 (80h18m27.663800847s) [noise=true , address=145.79.227.5:4242 , piece_id=54VY4XFFRB3G5P7VLHFAFUJQTVHVIEHKK2TZT4PNQUOUIFME7MFA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:145.79.227.5:4242): protocol: rpc: dial tcp 145.79.227.5:4242: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968005255 (80h18m27.663854008s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:126wqKRVvj3engw69RJSmdgdB91qMyG1Ed5XfUCpu3ZGq3uXSbe, address:145.79.227.37:4587): protocol: rpc: dial tcp 145.79.227.37:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968119682 (80h18m27.663968426s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: read tcp 188.166.225.170:54484->145.79.227.46:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968176942 (80h18m27.664025695s) [noise=true , address=145.79.227.43:4180 , piece_id=62QSVFJI44O5UMXM7SBZXY7SRMH6ZYSDD23OKDX5WXRLQCYNSXTQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: read tcp 188.166.225.170:48566->145.79.227.43:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968252796 (80h18m27.664101549s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1VzewCRbNpJv7kNwp9wruQuoVSaZFjA44qN33Ct7qm1GnfhtDj, address:145.79.227.45:4237): protocol: read tcp 188.166.225.170:60808->145.79.227.45:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968277563 (80h18m27.664126306s) [seg_pos={0 0}, upload=70665]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Nov 2 19:58:56.213414304 (80h18m27.909263047s) [upload=70665]: Done uploading segments... Nov 2 19:58:56.213452137 (80h18m27.90930088s) [upload=70665]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 5.647940348s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [............................XX..............................] duration: 3.395578263s trace: https://jaeger.dev.storj.io/trace/1a875a21fbc9493b FAIL (2025-11-02 20:22:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.395608691s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.697162711s trace: https://jaeger.dev.storj.io/trace/6469a32319fb23c3 FAIL (2025-11-02 20:22:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.939448529 (80h17m28.635297282s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940316662 (80h17m28.636165416s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940854095 (80h17m28.636702838s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941163254 (80h17m28.637011997s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941597421 (80h17m28.637446175s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.944379569 (80h17m28.640228322s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945350588 (80h17m28.641199341s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945999965 (80h17m28.641848708s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.946799317 (80h17m28.64264806s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949026458 (80h17m28.644875211s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949648072 (80h17m28.645496815s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:58.374326863 (80h17m30.070175616s) [upload=70665]: Next returned nil segment Nov 2 19:57:58.374346910 (80h17m30.070195674s) [upload=70665]: Waiting for error group managing segments... Nov 2 19:57:58.737256438 (80h17m30.433105181s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.932997019 (80h17m30.628845773s) [noise=true , address=45.91.116.8:4046 , piece_id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.948107422 (80h17m30.643956175s) [noise=true , address=77.90.31.19:4861 , piece_id=DPE7KWFAPMTW67GEMLT2YCBXVMLY75BAU7BLRRYN2UBDIWHUZNJQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.005265203 (80h17m30.701113946s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.052320256 (80h17m30.748169009s) [noise=true , address=77.90.31.13:4658 , piece_id=ZTX6WQ5M3TTA7KUKMJVKWHQHLMRMJW5EUUFVBRBQZDVBFZWWKZQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:58:55.966307021 (80h18m27.662155784s) [noise=true , address=145.79.227.3:4169 , piece_id=GCSPGUWUFDXU2TZ6BNUAEU75A4RUKYCRERUPQRD326RXSDAX7ATQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:145.79.227.3:4169): protocol: rpc: dial tcp 145.79.227.3:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966419525 (80h18m27.662268268s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12jfQbLaQjEXhVTbTPuawAQAbKJSN5RA8kBqz6JVDATQEGAsZ6K, address:145.79.227.13:4833): protocol: rpc: dial tcp 145.79.227.13:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966726459 (80h18m27.662575212s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: rpc: dial tcp 145.79.227.18:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966930668 (80h18m27.662779421s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1F5oTP56RsDAA7Yde8hYvq7Lx5tiFDUHoBvPrYnhQiFy2Rko3N, address:145.79.227.38:4995): protocol: read tcp 188.166.225.170:59394->145.79.227.38:4995: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967260457 (80h18m27.66310921s) [noise=true , address=145.79.227.41:4097 , piece_id=ANQYF4KYYUUGFNUOHEINXGP2VQATR2LNVY6L55FA5FOL5HNAJRGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:145.79.227.41:4097): protocol: rpc: dial tcp 145.79.227.41:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967696617 (80h18m27.66354536s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: rpc: dial tcp 145.79.227.48:4310: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967744338 (80h18m27.663593081s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:145.79.227.8:4357): protocol: rpc: dial tcp 145.79.227.8:4357: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967941153 (80h18m27.663789906s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12HohFM5Npi28Xq65JVMGUDK2bfX7xh5uYXNeHs3XW2VRH8rS72, address:145.79.227.49:4346): protocol: read tcp 188.166.225.170:47702->145.79.227.49:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967952104 (80h18m27.663800847s) [noise=true , address=145.79.227.5:4242 , piece_id=54VY4XFFRB3G5P7VLHFAFUJQTVHVIEHKK2TZT4PNQUOUIFME7MFA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:145.79.227.5:4242): protocol: rpc: dial tcp 145.79.227.5:4242: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968005255 (80h18m27.663854008s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:126wqKRVvj3engw69RJSmdgdB91qMyG1Ed5XfUCpu3ZGq3uXSbe, address:145.79.227.37:4587): protocol: rpc: dial tcp 145.79.227.37:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968119682 (80h18m27.663968426s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: read tcp 188.166.225.170:54484->145.79.227.46:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968176942 (80h18m27.664025695s) [noise=true , address=145.79.227.43:4180 , piece_id=62QSVFJI44O5UMXM7SBZXY7SRMH6ZYSDD23OKDX5WXRLQCYNSXTQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: read tcp 188.166.225.170:48566->145.79.227.43:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968252796 (80h18m27.664101549s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1VzewCRbNpJv7kNwp9wruQuoVSaZFjA44qN33Ct7qm1GnfhtDj, address:145.79.227.45:4237): protocol: read tcp 188.166.225.170:60808->145.79.227.45:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968277563 (80h18m27.664126306s) [seg_pos={0 0}, upload=70665]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Nov 2 19:58:56.213414304 (80h18m27.909263047s) [upload=70665]: Done uploading segments... Nov 2 19:58:56.213452137 (80h18m27.90930088s) [upload=70665]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 5.697194501s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 293.130402ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/42eaea040e9ca2bf FAIL (2025-11-02 20:22:12 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42676->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 293.162984ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 65.855462ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/580c04fa88dfbdac FAIL (2025-11-02 20:22:10 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:42672->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 65.877795ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.830822913s trace: https://jaeger.dev.storj.io/trace/6469a32319fb23c3 FAIL (2025-11-02 20:21:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.939448529 (80h17m28.635297282s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940316662 (80h17m28.636165416s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940854095 (80h17m28.636702838s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941163254 (80h17m28.637011997s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941597421 (80h17m28.637446175s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.944379569 (80h17m28.640228322s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945350588 (80h17m28.641199341s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945999965 (80h17m28.641848708s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.946799317 (80h17m28.64264806s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949026458 (80h17m28.644875211s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949648072 (80h17m28.645496815s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:58.374326863 (80h17m30.070175616s) [upload=70665]: Next returned nil segment Nov 2 19:57:58.374346910 (80h17m30.070195674s) [upload=70665]: Waiting for error group managing segments... Nov 2 19:57:58.737256438 (80h17m30.433105181s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.932997019 (80h17m30.628845773s) [noise=true , address=45.91.116.8:4046 , piece_id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.948107422 (80h17m30.643956175s) [noise=true , address=77.90.31.19:4861 , piece_id=DPE7KWFAPMTW67GEMLT2YCBXVMLY75BAU7BLRRYN2UBDIWHUZNJQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.005265203 (80h17m30.701113946s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.052320256 (80h17m30.748169009s) [noise=true , address=77.90.31.13:4658 , piece_id=ZTX6WQ5M3TTA7KUKMJVKWHQHLMRMJW5EUUFVBRBQZDVBFZWWKZQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:58:55.966307021 (80h18m27.662155784s) [noise=true , address=145.79.227.3:4169 , piece_id=GCSPGUWUFDXU2TZ6BNUAEU75A4RUKYCRERUPQRD326RXSDAX7ATQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:145.79.227.3:4169): protocol: rpc: dial tcp 145.79.227.3:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966419525 (80h18m27.662268268s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12jfQbLaQjEXhVTbTPuawAQAbKJSN5RA8kBqz6JVDATQEGAsZ6K, address:145.79.227.13:4833): protocol: rpc: dial tcp 145.79.227.13:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966726459 (80h18m27.662575212s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: rpc: dial tcp 145.79.227.18:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966930668 (80h18m27.662779421s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1F5oTP56RsDAA7Yde8hYvq7Lx5tiFDUHoBvPrYnhQiFy2Rko3N, address:145.79.227.38:4995): protocol: read tcp 188.166.225.170:59394->145.79.227.38:4995: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967260457 (80h18m27.66310921s) [noise=true , address=145.79.227.41:4097 , piece_id=ANQYF4KYYUUGFNUOHEINXGP2VQATR2LNVY6L55FA5FOL5HNAJRGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:145.79.227.41:4097): protocol: rpc: dial tcp 145.79.227.41:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967696617 (80h18m27.66354536s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: rpc: dial tcp 145.79.227.48:4310: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967744338 (80h18m27.663593081s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:145.79.227.8:4357): protocol: rpc: dial tcp 145.79.227.8:4357: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967941153 (80h18m27.663789906s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12HohFM5Npi28Xq65JVMGUDK2bfX7xh5uYXNeHs3XW2VRH8rS72, address:145.79.227.49:4346): protocol: read tcp 188.166.225.170:47702->145.79.227.49:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967952104 (80h18m27.663800847s) [noise=true , address=145.79.227.5:4242 , piece_id=54VY4XFFRB3G5P7VLHFAFUJQTVHVIEHKK2TZT4PNQUOUIFME7MFA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:145.79.227.5:4242): protocol: rpc: dial tcp 145.79.227.5:4242: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968005255 (80h18m27.663854008s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:126wqKRVvj3engw69RJSmdgdB91qMyG1Ed5XfUCpu3ZGq3uXSbe, address:145.79.227.37:4587): protocol: rpc: dial tcp 145.79.227.37:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968119682 (80h18m27.663968426s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: read tcp 188.166.225.170:54484->145.79.227.46:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968176942 (80h18m27.664025695s) [noise=true , address=145.79.227.43:4180 , piece_id=62QSVFJI44O5UMXM7SBZXY7SRMH6ZYSDD23OKDX5WXRLQCYNSXTQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: read tcp 188.166.225.170:48566->145.79.227.43:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968252796 (80h18m27.664101549s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1VzewCRbNpJv7kNwp9wruQuoVSaZFjA44qN33Ct7qm1GnfhtDj, address:145.79.227.45:4237): protocol: read tcp 188.166.225.170:60808->145.79.227.45:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968277563 (80h18m27.664126306s) [seg_pos={0 0}, upload=70665]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Nov 2 19:58:56.213414304 (80h18m27.909263047s) [upload=70665]: Done uploading segments... Nov 2 19:58:56.213452137 (80h18m27.90930088s) [upload=70665]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 5.830849424s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.652751281s trace: https://jaeger.dev.storj.io/trace/6469a32319fb23c3 FAIL (2025-11-02 20:21:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.939448529 (80h17m28.635297282s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940316662 (80h17m28.636165416s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940854095 (80h17m28.636702838s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941163254 (80h17m28.637011997s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941597421 (80h17m28.637446175s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.944379569 (80h17m28.640228322s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945350588 (80h17m28.641199341s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945999965 (80h17m28.641848708s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.946799317 (80h17m28.64264806s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949026458 (80h17m28.644875211s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949648072 (80h17m28.645496815s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:58.374326863 (80h17m30.070175616s) [upload=70665]: Next returned nil segment Nov 2 19:57:58.374346910 (80h17m30.070195674s) [upload=70665]: Waiting for error group managing segments... Nov 2 19:57:58.737256438 (80h17m30.433105181s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.932997019 (80h17m30.628845773s) [noise=true , address=45.91.116.8:4046 , piece_id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.948107422 (80h17m30.643956175s) [noise=true , address=77.90.31.19:4861 , piece_id=DPE7KWFAPMTW67GEMLT2YCBXVMLY75BAU7BLRRYN2UBDIWHUZNJQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.005265203 (80h17m30.701113946s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.052320256 (80h17m30.748169009s) [noise=true , address=77.90.31.13:4658 , piece_id=ZTX6WQ5M3TTA7KUKMJVKWHQHLMRMJW5EUUFVBRBQZDVBFZWWKZQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:58:55.966307021 (80h18m27.662155784s) [noise=true , address=145.79.227.3:4169 , piece_id=GCSPGUWUFDXU2TZ6BNUAEU75A4RUKYCRERUPQRD326RXSDAX7ATQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:145.79.227.3:4169): protocol: rpc: dial tcp 145.79.227.3:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966419525 (80h18m27.662268268s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12jfQbLaQjEXhVTbTPuawAQAbKJSN5RA8kBqz6JVDATQEGAsZ6K, address:145.79.227.13:4833): protocol: rpc: dial tcp 145.79.227.13:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966726459 (80h18m27.662575212s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: rpc: dial tcp 145.79.227.18:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966930668 (80h18m27.662779421s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1F5oTP56RsDAA7Yde8hYvq7Lx5tiFDUHoBvPrYnhQiFy2Rko3N, address:145.79.227.38:4995): protocol: read tcp 188.166.225.170:59394->145.79.227.38:4995: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967260457 (80h18m27.66310921s) [noise=true , address=145.79.227.41:4097 , piece_id=ANQYF4KYYUUGFNUOHEINXGP2VQATR2LNVY6L55FA5FOL5HNAJRGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:145.79.227.41:4097): protocol: rpc: dial tcp 145.79.227.41:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967696617 (80h18m27.66354536s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: rpc: dial tcp 145.79.227.48:4310: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967744338 (80h18m27.663593081s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:145.79.227.8:4357): protocol: rpc: dial tcp 145.79.227.8:4357: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967941153 (80h18m27.663789906s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12HohFM5Npi28Xq65JVMGUDK2bfX7xh5uYXNeHs3XW2VRH8rS72, address:145.79.227.49:4346): protocol: read tcp 188.166.225.170:47702->145.79.227.49:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967952104 (80h18m27.663800847s) [noise=true , address=145.79.227.5:4242 , piece_id=54VY4XFFRB3G5P7VLHFAFUJQTVHVIEHKK2TZT4PNQUOUIFME7MFA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:145.79.227.5:4242): protocol: rpc: dial tcp 145.79.227.5:4242: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968005255 (80h18m27.663854008s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:126wqKRVvj3engw69RJSmdgdB91qMyG1Ed5XfUCpu3ZGq3uXSbe, address:145.79.227.37:4587): protocol: rpc: dial tcp 145.79.227.37:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968119682 (80h18m27.663968426s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: read tcp 188.166.225.170:54484->145.79.227.46:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968176942 (80h18m27.664025695s) [noise=true , address=145.79.227.43:4180 , piece_id=62QSVFJI44O5UMXM7SBZXY7SRMH6ZYSDD23OKDX5WXRLQCYNSXTQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: read tcp 188.166.225.170:48566->145.79.227.43:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968252796 (80h18m27.664101549s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1VzewCRbNpJv7kNwp9wruQuoVSaZFjA44qN33Ct7qm1GnfhtDj, address:145.79.227.45:4237): protocol: read tcp 188.166.225.170:60808->145.79.227.45:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968277563 (80h18m27.664126306s) [seg_pos={0 0}, upload=70665]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Nov 2 19:58:56.213414304 (80h18m27.909263047s) [upload=70665]: Done uploading segments... Nov 2 19:58:56.213452137 (80h18m27.90930088s) [upload=70665]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 5.652780175s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 308.610741ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/42eaea040e9ca2bf FAIL (2025-11-02 20:21:41 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42676->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 308.640837ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 53.146142ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/580c04fa88dfbdac FAIL (2025-11-02 20:21:38 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:42672->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 53.176159ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.............................XX.............................] duration: 3.429596469s trace: https://jaeger.dev.storj.io/trace/1a875a21fbc9493b FAIL (2025-11-02 20:21:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.429609486s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 57.813529ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/580c04fa88dfbdac FAIL (2025-11-02 20:21:11 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:42672->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 57.868052ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..............................XX............................] duration: 3.440130411s trace: https://jaeger.dev.storj.io/trace/1a875a21fbc9493b FAIL (2025-11-02 20:20:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.440160951s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 316.742205ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/42eaea040e9ca2bf FAIL (2025-11-02 20:20:54 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42676->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 316.77174ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.655633319s trace: https://jaeger.dev.storj.io/trace/6469a32319fb23c3 FAIL (2025-11-02 20:20:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.939448529 (80h17m28.635297282s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940316662 (80h17m28.636165416s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940854095 (80h17m28.636702838s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941163254 (80h17m28.637011997s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941597421 (80h17m28.637446175s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.944379569 (80h17m28.640228322s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945350588 (80h17m28.641199341s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945999965 (80h17m28.641848708s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.946799317 (80h17m28.64264806s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949026458 (80h17m28.644875211s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949648072 (80h17m28.645496815s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:58.374326863 (80h17m30.070175616s) [upload=70665]: Next returned nil segment Nov 2 19:57:58.374346910 (80h17m30.070195674s) [upload=70665]: Waiting for error group managing segments... Nov 2 19:57:58.737256438 (80h17m30.433105181s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.932997019 (80h17m30.628845773s) [noise=true , address=45.91.116.8:4046 , piece_id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.948107422 (80h17m30.643956175s) [noise=true , address=77.90.31.19:4861 , piece_id=DPE7KWFAPMTW67GEMLT2YCBXVMLY75BAU7BLRRYN2UBDIWHUZNJQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.005265203 (80h17m30.701113946s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.052320256 (80h17m30.748169009s) [noise=true , address=77.90.31.13:4658 , piece_id=ZTX6WQ5M3TTA7KUKMJVKWHQHLMRMJW5EUUFVBRBQZDVBFZWWKZQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:58:55.966307021 (80h18m27.662155784s) [noise=true , address=145.79.227.3:4169 , piece_id=GCSPGUWUFDXU2TZ6BNUAEU75A4RUKYCRERUPQRD326RXSDAX7ATQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:145.79.227.3:4169): protocol: rpc: dial tcp 145.79.227.3:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966419525 (80h18m27.662268268s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12jfQbLaQjEXhVTbTPuawAQAbKJSN5RA8kBqz6JVDATQEGAsZ6K, address:145.79.227.13:4833): protocol: rpc: dial tcp 145.79.227.13:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966726459 (80h18m27.662575212s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: rpc: dial tcp 145.79.227.18:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966930668 (80h18m27.662779421s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1F5oTP56RsDAA7Yde8hYvq7Lx5tiFDUHoBvPrYnhQiFy2Rko3N, address:145.79.227.38:4995): protocol: read tcp 188.166.225.170:59394->145.79.227.38:4995: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967260457 (80h18m27.66310921s) [noise=true , address=145.79.227.41:4097 , piece_id=ANQYF4KYYUUGFNUOHEINXGP2VQATR2LNVY6L55FA5FOL5HNAJRGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:145.79.227.41:4097): protocol: rpc: dial tcp 145.79.227.41:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967696617 (80h18m27.66354536s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: rpc: dial tcp 145.79.227.48:4310: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967744338 (80h18m27.663593081s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:145.79.227.8:4357): protocol: rpc: dial tcp 145.79.227.8:4357: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967941153 (80h18m27.663789906s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12HohFM5Npi28Xq65JVMGUDK2bfX7xh5uYXNeHs3XW2VRH8rS72, address:145.79.227.49:4346): protocol: read tcp 188.166.225.170:47702->145.79.227.49:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967952104 (80h18m27.663800847s) [noise=true , address=145.79.227.5:4242 , piece_id=54VY4XFFRB3G5P7VLHFAFUJQTVHVIEHKK2TZT4PNQUOUIFME7MFA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:145.79.227.5:4242): protocol: rpc: dial tcp 145.79.227.5:4242: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968005255 (80h18m27.663854008s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:126wqKRVvj3engw69RJSmdgdB91qMyG1Ed5XfUCpu3ZGq3uXSbe, address:145.79.227.37:4587): protocol: rpc: dial tcp 145.79.227.37:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968119682 (80h18m27.663968426s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: read tcp 188.166.225.170:54484->145.79.227.46:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968176942 (80h18m27.664025695s) [noise=true , address=145.79.227.43:4180 , piece_id=62QSVFJI44O5UMXM7SBZXY7SRMH6ZYSDD23OKDX5WXRLQCYNSXTQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: read tcp 188.166.225.170:48566->145.79.227.43:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968252796 (80h18m27.664101549s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1VzewCRbNpJv7kNwp9wruQuoVSaZFjA44qN33Ct7qm1GnfhtDj, address:145.79.227.45:4237): protocol: read tcp 188.166.225.170:60808->145.79.227.45:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968277563 (80h18m27.664126306s) [seg_pos={0 0}, upload=70665]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Nov 2 19:58:56.213414304 (80h18m27.909263047s) [upload=70665]: Done uploading segments... Nov 2 19:58:56.213452137 (80h18m27.90930088s) [upload=70665]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 5.655659117s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 49.757509ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/580c04fa88dfbdac FAIL (2025-11-02 20:20:33 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:42672->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 49.770824ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...............................XX...........................] duration: 3.490970378s trace: https://jaeger.dev.storj.io/trace/1a875a21fbc9493b FAIL (2025-11-02 20:20:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.490985927s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [................................XX..........................] duration: 3.399994062s trace: https://jaeger.dev.storj.io/trace/1a875a21fbc9493b FAIL (2025-11-02 20:20:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.400175017s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 358.217961ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/42eaea040e9ca2bf FAIL (2025-11-02 20:19:59 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42676->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 358.24361ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 54.2638ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/580c04fa88dfbdac FAIL (2025-11-02 20:19:56 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:42672->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 54.291925ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 307.778527ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/42eaea040e9ca2bf FAIL (2025-11-02 20:19:34 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42676->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 307.802182ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.618659894s trace: https://jaeger.dev.storj.io/trace/6469a32319fb23c3 FAIL (2025-11-02 20:19:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.939448529 (80h17m28.635297282s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940316662 (80h17m28.636165416s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940854095 (80h17m28.636702838s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941163254 (80h17m28.637011997s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941597421 (80h17m28.637446175s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.944379569 (80h17m28.640228322s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945350588 (80h17m28.641199341s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945999965 (80h17m28.641848708s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.946799317 (80h17m28.64264806s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949026458 (80h17m28.644875211s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949648072 (80h17m28.645496815s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:58.374326863 (80h17m30.070175616s) [upload=70665]: Next returned nil segment Nov 2 19:57:58.374346910 (80h17m30.070195674s) [upload=70665]: Waiting for error group managing segments... Nov 2 19:57:58.737256438 (80h17m30.433105181s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.932997019 (80h17m30.628845773s) [noise=true , address=45.91.116.8:4046 , piece_id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.948107422 (80h17m30.643956175s) [noise=true , address=77.90.31.19:4861 , piece_id=DPE7KWFAPMTW67GEMLT2YCBXVMLY75BAU7BLRRYN2UBDIWHUZNJQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.005265203 (80h17m30.701113946s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.052320256 (80h17m30.748169009s) [noise=true , address=77.90.31.13:4658 , piece_id=ZTX6WQ5M3TTA7KUKMJVKWHQHLMRMJW5EUUFVBRBQZDVBFZWWKZQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:58:55.966307021 (80h18m27.662155784s) [noise=true , address=145.79.227.3:4169 , piece_id=GCSPGUWUFDXU2TZ6BNUAEU75A4RUKYCRERUPQRD326RXSDAX7ATQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:145.79.227.3:4169): protocol: rpc: dial tcp 145.79.227.3:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966419525 (80h18m27.662268268s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12jfQbLaQjEXhVTbTPuawAQAbKJSN5RA8kBqz6JVDATQEGAsZ6K, address:145.79.227.13:4833): protocol: rpc: dial tcp 145.79.227.13:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966726459 (80h18m27.662575212s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: rpc: dial tcp 145.79.227.18:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966930668 (80h18m27.662779421s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1F5oTP56RsDAA7Yde8hYvq7Lx5tiFDUHoBvPrYnhQiFy2Rko3N, address:145.79.227.38:4995): protocol: read tcp 188.166.225.170:59394->145.79.227.38:4995: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967260457 (80h18m27.66310921s) [noise=true , address=145.79.227.41:4097 , piece_id=ANQYF4KYYUUGFNUOHEINXGP2VQATR2LNVY6L55FA5FOL5HNAJRGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:145.79.227.41:4097): protocol: rpc: dial tcp 145.79.227.41:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967696617 (80h18m27.66354536s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: rpc: dial tcp 145.79.227.48:4310: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967744338 (80h18m27.663593081s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:145.79.227.8:4357): protocol: rpc: dial tcp 145.79.227.8:4357: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967941153 (80h18m27.663789906s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12HohFM5Npi28Xq65JVMGUDK2bfX7xh5uYXNeHs3XW2VRH8rS72, address:145.79.227.49:4346): protocol: read tcp 188.166.225.170:47702->145.79.227.49:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967952104 (80h18m27.663800847s) [noise=true , address=145.79.227.5:4242 , piece_id=54VY4XFFRB3G5P7VLHFAFUJQTVHVIEHKK2TZT4PNQUOUIFME7MFA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:145.79.227.5:4242): protocol: rpc: dial tcp 145.79.227.5:4242: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968005255 (80h18m27.663854008s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:126wqKRVvj3engw69RJSmdgdB91qMyG1Ed5XfUCpu3ZGq3uXSbe, address:145.79.227.37:4587): protocol: rpc: dial tcp 145.79.227.37:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968119682 (80h18m27.663968426s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: read tcp 188.166.225.170:54484->145.79.227.46:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968176942 (80h18m27.664025695s) [noise=true , address=145.79.227.43:4180 , piece_id=62QSVFJI44O5UMXM7SBZXY7SRMH6ZYSDD23OKDX5WXRLQCYNSXTQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: read tcp 188.166.225.170:48566->145.79.227.43:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968252796 (80h18m27.664101549s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1VzewCRbNpJv7kNwp9wruQuoVSaZFjA44qN33Ct7qm1GnfhtDj, address:145.79.227.45:4237): protocol: read tcp 188.166.225.170:60808->145.79.227.45:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968277563 (80h18m27.664126306s) [seg_pos={0 0}, upload=70665]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Nov 2 19:58:56.213414304 (80h18m27.909263047s) [upload=70665]: Done uploading segments... Nov 2 19:58:56.213452137 (80h18m27.90930088s) [upload=70665]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 5.618687437s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.................................XX.........................] duration: 3.466131753s trace: https://jaeger.dev.storj.io/trace/1a875a21fbc9493b FAIL (2025-11-02 20:18:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.466184613s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 44.829464ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/580c04fa88dfbdac FAIL (2025-11-02 20:18:45 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:42672->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 44.854382ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.689551532s trace: https://jaeger.dev.storj.io/trace/6469a32319fb23c3 FAIL (2025-11-02 20:18:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.939448529 (80h17m28.635297282s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940316662 (80h17m28.636165416s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940854095 (80h17m28.636702838s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941163254 (80h17m28.637011997s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941597421 (80h17m28.637446175s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.944379569 (80h17m28.640228322s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945350588 (80h17m28.641199341s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945999965 (80h17m28.641848708s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.946799317 (80h17m28.64264806s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949026458 (80h17m28.644875211s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949648072 (80h17m28.645496815s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:58.374326863 (80h17m30.070175616s) [upload=70665]: Next returned nil segment Nov 2 19:57:58.374346910 (80h17m30.070195674s) [upload=70665]: Waiting for error group managing segments... Nov 2 19:57:58.737256438 (80h17m30.433105181s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.932997019 (80h17m30.628845773s) [noise=true , address=45.91.116.8:4046 , piece_id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.948107422 (80h17m30.643956175s) [noise=true , address=77.90.31.19:4861 , piece_id=DPE7KWFAPMTW67GEMLT2YCBXVMLY75BAU7BLRRYN2UBDIWHUZNJQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.005265203 (80h17m30.701113946s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.052320256 (80h17m30.748169009s) [noise=true , address=77.90.31.13:4658 , piece_id=ZTX6WQ5M3TTA7KUKMJVKWHQHLMRMJW5EUUFVBRBQZDVBFZWWKZQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:58:55.966307021 (80h18m27.662155784s) [noise=true , address=145.79.227.3:4169 , piece_id=GCSPGUWUFDXU2TZ6BNUAEU75A4RUKYCRERUPQRD326RXSDAX7ATQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:145.79.227.3:4169): protocol: rpc: dial tcp 145.79.227.3:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966419525 (80h18m27.662268268s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12jfQbLaQjEXhVTbTPuawAQAbKJSN5RA8kBqz6JVDATQEGAsZ6K, address:145.79.227.13:4833): protocol: rpc: dial tcp 145.79.227.13:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966726459 (80h18m27.662575212s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: rpc: dial tcp 145.79.227.18:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966930668 (80h18m27.662779421s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1F5oTP56RsDAA7Yde8hYvq7Lx5tiFDUHoBvPrYnhQiFy2Rko3N, address:145.79.227.38:4995): protocol: read tcp 188.166.225.170:59394->145.79.227.38:4995: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967260457 (80h18m27.66310921s) [noise=true , address=145.79.227.41:4097 , piece_id=ANQYF4KYYUUGFNUOHEINXGP2VQATR2LNVY6L55FA5FOL5HNAJRGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:145.79.227.41:4097): protocol: rpc: dial tcp 145.79.227.41:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967696617 (80h18m27.66354536s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: rpc: dial tcp 145.79.227.48:4310: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967744338 (80h18m27.663593081s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:145.79.227.8:4357): protocol: rpc: dial tcp 145.79.227.8:4357: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967941153 (80h18m27.663789906s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12HohFM5Npi28Xq65JVMGUDK2bfX7xh5uYXNeHs3XW2VRH8rS72, address:145.79.227.49:4346): protocol: read tcp 188.166.225.170:47702->145.79.227.49:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967952104 (80h18m27.663800847s) [noise=true , address=145.79.227.5:4242 , piece_id=54VY4XFFRB3G5P7VLHFAFUJQTVHVIEHKK2TZT4PNQUOUIFME7MFA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:145.79.227.5:4242): protocol: rpc: dial tcp 145.79.227.5:4242: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968005255 (80h18m27.663854008s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:126wqKRVvj3engw69RJSmdgdB91qMyG1Ed5XfUCpu3ZGq3uXSbe, address:145.79.227.37:4587): protocol: rpc: dial tcp 145.79.227.37:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968119682 (80h18m27.663968426s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: read tcp 188.166.225.170:54484->145.79.227.46:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968176942 (80h18m27.664025695s) [noise=true , address=145.79.227.43:4180 , piece_id=62QSVFJI44O5UMXM7SBZXY7SRMH6ZYSDD23OKDX5WXRLQCYNSXTQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: read tcp 188.166.225.170:48566->145.79.227.43:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968252796 (80h18m27.664101549s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1VzewCRbNpJv7kNwp9wruQuoVSaZFjA44qN33Ct7qm1GnfhtDj, address:145.79.227.45:4237): protocol: read tcp 188.166.225.170:60808->145.79.227.45:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968277563 (80h18m27.664126306s) [seg_pos={0 0}, upload=70665]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Nov 2 19:58:56.213414304 (80h18m27.909263047s) [upload=70665]: Done uploading segments... Nov 2 19:58:56.213452137 (80h18m27.90930088s) [upload=70665]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 5.689585177s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 590.754289ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/42eaea040e9ca2bf FAIL (2025-11-02 20:18:37 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42676->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 590.781581ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.686977434s trace: https://jaeger.dev.storj.io/trace/6469a32319fb23c3 FAIL (2025-11-02 20:18:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 5) ======LOGS====== _id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.939448529 (80h17m28.635297282s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940316662 (80h17m28.636165416s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.940854095 (80h17m28.636702838s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941163254 (80h17m28.637011997s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.941597421 (80h17m28.637446175s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.944379569 (80h17m28.640228322s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945350588 (80h17m28.641199341s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.945999965 (80h17m28.641848708s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.946799317 (80h17m28.64264806s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949026458 (80h17m28.644875211s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:56.949648072 (80h17m28.645496815s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Uploading piece... Nov 2 19:57:58.374326863 (80h17m30.070175616s) [upload=70665]: Next returned nil segment Nov 2 19:57:58.374346910 (80h17m30.070195674s) [upload=70665]: Waiting for error group managing segments... Nov 2 19:57:58.737256438 (80h17m30.433105181s) [noise=true , address=64.49.13.169:4715 , piece_id=MSNF6WCWWVJEM5X5FDCH5OFWJENAIQQ6FBY63GGC3NG7HYR66X6A, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.932997019 (80h17m30.628845773s) [noise=true , address=45.91.116.8:4046 , piece_id=USBTO6WYEAFPBP3NY7PKZZWDDPLI2QBWX33ZHTJVX6XWBGGWKXWA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:58.948107422 (80h17m30.643956175s) [noise=true , address=77.90.31.19:4861 , piece_id=DPE7KWFAPMTW67GEMLT2YCBXVMLY75BAU7BLRRYN2UBDIWHUZNJQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.005265203 (80h17m30.701113946s) [noise=true , address=77.90.31.11:4571 , piece_id=Z6UVBULYAS4O4FUHFRFGZ3YD2D24BMAJFZABU6HCDG2GHXZF6TWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:57:59.052320256 (80h17m30.748169009s) [noise=true , address=77.90.31.13:4658 , piece_id=ZTX6WQ5M3TTA7KUKMJVKWHQHLMRMJW5EUUFVBRBQZDVBFZWWKZQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: <nil> Nov 2 19:58:55.966307021 (80h18m27.662155784s) [noise=true , address=145.79.227.3:4169 , piece_id=GCSPGUWUFDXU2TZ6BNUAEU75A4RUKYCRERUPQRD326RXSDAX7ATQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12AcJsPCQkjCgLxMTVcjMC5BpEe9RqzcC4CNC3oJDUt4f8cTbNX, address:145.79.227.3:4169): protocol: rpc: dial tcp 145.79.227.3:4169: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966419525 (80h18m27.662268268s) [noise=true , address=145.79.227.13:4833 , piece_id=V62DR4BLHZXAO6H2AYYEQ2L42TKBRCRD5ECU32RRVNTEKTJPTPWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12jfQbLaQjEXhVTbTPuawAQAbKJSN5RA8kBqz6JVDATQEGAsZ6K, address:145.79.227.13:4833): protocol: rpc: dial tcp 145.79.227.13:4833: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966726459 (80h18m27.662575212s) [noise=true , address=145.79.227.18:4030 , piece_id=OMSVZ6FODSRR3BBORZ7DSFFE7AYSSIUEIUOO4KMCXYOE2FM6DPGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1MGCEjVE57ej5VuXgC6pYPFNUuAL851p2gqC53xxj6hEXvgYM5, address:145.79.227.18:4030): protocol: rpc: dial tcp 145.79.227.18:4030: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.966930668 (80h18m27.662779421s) [noise=true , address=145.79.227.38:4995 , piece_id=27YBF4I34SVRPINHA53UNKNCHIQVRUH7C4E2SIJIXEAVXCBWYV6Q, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1F5oTP56RsDAA7Yde8hYvq7Lx5tiFDUHoBvPrYnhQiFy2Rko3N, address:145.79.227.38:4995): protocol: read tcp 188.166.225.170:59394->145.79.227.38:4995: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967260457 (80h18m27.66310921s) [noise=true , address=145.79.227.41:4097 , piece_id=ANQYF4KYYUUGFNUOHEINXGP2VQATR2LNVY6L55FA5FOL5HNAJRGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1sMhTktqf2H4aZSyFV3m9Z6eMVxL2U52bvUHVHUh2gUCVp32AM, address:145.79.227.41:4097): protocol: rpc: dial tcp 145.79.227.41:4097: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967696617 (80h18m27.66354536s) [noise=true , address=145.79.227.48:4310 , piece_id=QL3QV64VFPDFTM525S6Q5MPBHYRJ5ASE56GQLB3425X6VT5S7NQA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12uYJ3ag6xFbLiu39t7jqwvtbo1QLwHw7k9PmfrMWBZBappp8MX, address:145.79.227.48:4310): protocol: rpc: dial tcp 145.79.227.48:4310: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967744338 (80h18m27.663593081s) [noise=true , address=145.79.227.8:4357 , piece_id=FZDQG5JYXGRHS4QPMXBHZCOC5TRAC6TUEGN5W3XM2ICCNGZIPBDA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1K1edm26dQy31zzdfsqmrnAdgJ28EJZiiKj1GzpeCMwEBYWEp9, address:145.79.227.8:4357): protocol: rpc: dial tcp 145.79.227.8:4357: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967941153 (80h18m27.663789906s) [noise=true , address=145.79.227.49:4346 , piece_id=3VOF5UY4UZZ2F3YKWLQZAMPROUQHDZZGEMERLV2CPXH6M3ITCMJA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:12HohFM5Npi28Xq65JVMGUDK2bfX7xh5uYXNeHs3XW2VRH8rS72, address:145.79.227.49:4346): protocol: read tcp 188.166.225.170:47702->145.79.227.49:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.967952104 (80h18m27.663800847s) [noise=true , address=145.79.227.5:4242 , piece_id=54VY4XFFRB3G5P7VLHFAFUJQTVHVIEHKK2TZT4PNQUOUIFME7MFA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1inR8rjUp4fRQWcaD2S7MRxveHyFxKoutWoEiuKQG3C895PaMz, address:145.79.227.5:4242): protocol: rpc: dial tcp 145.79.227.5:4242: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968005255 (80h18m27.663854008s) [noise=true , address=145.79.227.37:4587 , piece_id=OI6L5A2N5RIWHFOVROUJLJXWIRILCPRL6DCOK5KBJE57RUTZFICQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:126wqKRVvj3engw69RJSmdgdB91qMyG1Ed5XfUCpu3ZGq3uXSbe, address:145.79.227.37:4587): protocol: rpc: dial tcp 145.79.227.37:4587: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968119682 (80h18m27.663968426s) [noise=true , address=145.79.227.46:4602 , piece_id=SVDV3T5Y57JACYZWRIX7AMSE3OVXDXNZEXE5LDY47PFUWOKOCRWQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1kgUqvw1r3gSKw1bNjzys3AiAJXdkAnSafmvcPgNVPBbN9JS1d, address:145.79.227.46:4602): protocol: read tcp 188.166.225.170:54484->145.79.227.46:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968176942 (80h18m27.664025695s) [noise=true , address=145.79.227.43:4180 , piece_id=62QSVFJI44O5UMXM7SBZXY7SRMH6ZYSDD23OKDX5WXRLQCYNSXTQ, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1rhkcipPBZNFScBjBhWdigzSMwENSmzhCucnC2iuyc9udpweE3, address:145.79.227.43:4180): protocol: read tcp 188.166.225.170:48566->145.79.227.43:4180: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968252796 (80h18m27.664101549s) [noise=true , address=145.79.227.45:4237 , piece_id=4IFOBDAZWXHENJ4ZYTBC2RFFZ74ZI4QXWEL3SQS655NE2M3JXHGA, seg_pos={0 0}, upload=70665]: Done uploading piece. err: ecclient: upload failed (node:1VzewCRbNpJv7kNwp9wruQuoVSaZFjA44qN33Ct7qm1GnfhtDj, address:145.79.227.45:4237): protocol: read tcp 188.166.225.170:60808->145.79.227.45:4237: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 2 19:58:55.968277563 (80h18m27.664126306s) [seg_pos={0 0}, upload=70665]: Done waiting for segment. successful: 5 optimal: 14 errs: <nil> Nov 2 19:58:56.213414304 (80h18m27.909263047s) [upload=70665]: Done uploading segments... Nov 2 19:58:56.213452137 (80h18m27.90930088s) [upload=70665]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 5) Duration: 5.687004858s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)