Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................XX.....X.............] duration: 8.577765337s trace: https://jaeger.dev.storj.io/trace/6ba329aed526607a FAIL (2025-11-11 16:23:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== J6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.204239805 (56h9m48.755710266s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205024630 (56h9m48.756495091s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205617369 (56h9m48.75708783s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.206216129 (56h9m48.75768659s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.207869137 (56h9m48.759339598s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.209417975 (56h9m48.760888436s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.210891231 (56h9m48.762361693s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.212264837 (56h9m48.763735288s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.213197944 (56h9m48.764668405s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.217083202 (56h9m48.768553663s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.218920661 (56h9m48.770391122s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.220461635 (56h9m48.771932086s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.221453104 (56h9m48.772923565s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.222919085 (56h9m48.774389576s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.224462674 (56h9m48.775933135s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:51.977601348 (56h9m50.52907181s) [upload=49489]: Next returned nil segment Nov 11 16:13:51.977618881 (56h9m50.529089342s) [upload=49489]: Waiting for error group managing segments... Nov 11 16:13:52.318041740 (56h9m50.869512201s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.319228510 (56h9m50.870699001s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.330974096 (56h9m50.882444547s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.366779437 (56h9m50.918249898s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.403498277 (56h9m50.954968738s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.410852602 (56h9m50.962323093s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.441652722 (56h9m50.993123183s) [noise=true , address=77.90.31.15:4698 , piece_id=KJZ2GOHKXXWREHUAIMSJ6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.943383035 (56h9m51.494853496s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:14:19.226465189 (56h10m17.77793565s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection Nov 11 16:14:19.226675380 (56h10m17.778145851s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection Nov 11 16:14:19.226719734 (56h10m17.778190195s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection Nov 11 16:14:19.226939854 (56h10m17.778410325s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: read tcp 188.166.225.170:46848->145.79.227.27:4689: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.226954371 (56h10m17.778424832s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection Nov 11 16:14:19.227194228 (56h10m17.778664699s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:47846->145.79.227.32:4834: use of closed network connection; context deadline exceeded Nov 11 16:14:19.227398437 (56h10m17.778868898s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:46890->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.228143486 (56h10m17.779614047s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection Nov 11 16:14:19.228201325 (56h10m17.779671797s) [seg_pos={0 0}, upload=49489]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 11 16:14:19.496444177 (56h10m18.047914638s) [upload=49489]: Done uploading segments... Nov 11 16:14:19.496495415 (56h10m18.047965877s) [upload=49489]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 8.577797739s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.380552901s trace: https://jaeger.dev.storj.io/trace/44d8b784072367e4 FAIL (2025-11-11 16:23:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== iece... Nov 11 16:08:48.413266786 (56h4m46.964737247s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.413968803 (56h4m46.965439264s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414381078 (56h4m46.965851539s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414796339 (56h4m46.9662668s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415222971 (56h4m46.966693432s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415704538 (56h4m46.967174989s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416081355 (56h4m46.967551816s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416488901 (56h4m46.967959362s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416811835 (56h4m46.968282287s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417126284 (56h4m46.968596735s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417504084 (56h4m46.968974535s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.418850267 (56h4m46.970320719s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.419254357 (56h4m46.970724818s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.421629780 (56h4m46.973100241s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.423823017 (56h4m46.975293478s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.426626956 (56h4m46.978097417s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.428526082 (56h4m46.979996543s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:49.743855894 (56h4m48.295326345s) [upload=49413]: Next returned nil segment Nov 11 16:08:49.743869990 (56h4m48.295340451s) [upload=49413]: Waiting for error group managing segments... Nov 11 16:08:50.083885022 (56h4m48.635355483s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.099619755 (56h4m48.651090216s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.185889601 (56h4m48.737360062s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.210908903 (56h4m48.762379354s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.233314608 (56h4m48.784785069s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.385166156 (56h4m48.936636617s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.412131434 (56h4m48.963601895s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.421729551 (56h4m48.973200012s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.430205912 (56h4m48.981676373s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.455736998 (56h4m49.007207459s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.497672944 (56h4m49.049143395s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.511232905 (56h4m49.062703366s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.569351094 (56h4m49.120821555s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:09:47.409293414 (56h5m45.960763875s) [noise=true , address=145.79.227.17:4994 , piece_id=ALC7PHRXP3EPN37WJZ5K4CF7SENX2T2T4GZZVEGCKO6EUXHL4GYA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:13nneaPpsRT7mVEc6cDt7GiZSMTU68Nnk1rr9Lbn5ycZkmQGrB, address:145.79.227.17:4994): protocol: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection Nov 11 16:09:47.409343289 (56h5m45.96081375s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection Nov 11 16:09:47.409602432 (56h5m45.961072903s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T, address:145.79.227.27:4707): protocol: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection Nov 11 16:09:47.409644141 (56h5m45.961114602s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection Nov 11 16:09:47.409737079 (56h5m45.96120753s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:12NBsd33JtAwPdcbD8vBzZUYpjiqZpRzs8y2QrYJFE245pd8T5X, address:145.79.227.44:4228): protocol: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection Nov 11 16:09:47.409766335 (56h5m45.961236786s) [seg_pos={0 0}, upload=49413]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Nov 11 16:09:47.658593534 (56h5m46.210063996s) [upload=49413]: Done uploading segments... Nov 11 16:09:47.658685831 (56h5m46.210156282s) [upload=49413]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.380579462s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 68.362126ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/79b21962f2d8e182 FAIL (2025-11-11 16:23:12 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:55382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 68.383126ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.062032568s trace: https://jaeger.dev.storj.io/trace/44d8b784072367e4 FAIL (2025-11-11 16:23:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== iece... Nov 11 16:08:48.413266786 (56h4m46.964737247s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.413968803 (56h4m46.965439264s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414381078 (56h4m46.965851539s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414796339 (56h4m46.9662668s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415222971 (56h4m46.966693432s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415704538 (56h4m46.967174989s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416081355 (56h4m46.967551816s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416488901 (56h4m46.967959362s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416811835 (56h4m46.968282287s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417126284 (56h4m46.968596735s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417504084 (56h4m46.968974535s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.418850267 (56h4m46.970320719s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.419254357 (56h4m46.970724818s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.421629780 (56h4m46.973100241s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.423823017 (56h4m46.975293478s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.426626956 (56h4m46.978097417s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.428526082 (56h4m46.979996543s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:49.743855894 (56h4m48.295326345s) [upload=49413]: Next returned nil segment Nov 11 16:08:49.743869990 (56h4m48.295340451s) [upload=49413]: Waiting for error group managing segments... Nov 11 16:08:50.083885022 (56h4m48.635355483s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.099619755 (56h4m48.651090216s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.185889601 (56h4m48.737360062s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.210908903 (56h4m48.762379354s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.233314608 (56h4m48.784785069s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.385166156 (56h4m48.936636617s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.412131434 (56h4m48.963601895s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.421729551 (56h4m48.973200012s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.430205912 (56h4m48.981676373s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.455736998 (56h4m49.007207459s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.497672944 (56h4m49.049143395s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.511232905 (56h4m49.062703366s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.569351094 (56h4m49.120821555s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:09:47.409293414 (56h5m45.960763875s) [noise=true , address=145.79.227.17:4994 , piece_id=ALC7PHRXP3EPN37WJZ5K4CF7SENX2T2T4GZZVEGCKO6EUXHL4GYA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:13nneaPpsRT7mVEc6cDt7GiZSMTU68Nnk1rr9Lbn5ycZkmQGrB, address:145.79.227.17:4994): protocol: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection Nov 11 16:09:47.409343289 (56h5m45.96081375s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection Nov 11 16:09:47.409602432 (56h5m45.961072903s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T, address:145.79.227.27:4707): protocol: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection Nov 11 16:09:47.409644141 (56h5m45.961114602s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection Nov 11 16:09:47.409737079 (56h5m45.96120753s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:12NBsd33JtAwPdcbD8vBzZUYpjiqZpRzs8y2QrYJFE245pd8T5X, address:145.79.227.44:4228): protocol: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection Nov 11 16:09:47.409766335 (56h5m45.961236786s) [seg_pos={0 0}, upload=49413]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Nov 11 16:09:47.658593534 (56h5m46.210063996s) [upload=49413]: Done uploading segments... Nov 11 16:09:47.658685831 (56h5m46.210156282s) [upload=49413]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.062063306s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................XX.....X............] duration: 6.903164676s trace: https://jaeger.dev.storj.io/trace/6ba329aed526607a FAIL (2025-11-11 16:22:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== J6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.204239805 (56h9m48.755710266s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205024630 (56h9m48.756495091s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205617369 (56h9m48.75708783s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.206216129 (56h9m48.75768659s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.207869137 (56h9m48.759339598s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.209417975 (56h9m48.760888436s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.210891231 (56h9m48.762361693s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.212264837 (56h9m48.763735288s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.213197944 (56h9m48.764668405s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.217083202 (56h9m48.768553663s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.218920661 (56h9m48.770391122s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.220461635 (56h9m48.771932086s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.221453104 (56h9m48.772923565s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.222919085 (56h9m48.774389576s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.224462674 (56h9m48.775933135s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:51.977601348 (56h9m50.52907181s) [upload=49489]: Next returned nil segment Nov 11 16:13:51.977618881 (56h9m50.529089342s) [upload=49489]: Waiting for error group managing segments... Nov 11 16:13:52.318041740 (56h9m50.869512201s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.319228510 (56h9m50.870699001s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.330974096 (56h9m50.882444547s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.366779437 (56h9m50.918249898s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.403498277 (56h9m50.954968738s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.410852602 (56h9m50.962323093s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.441652722 (56h9m50.993123183s) [noise=true , address=77.90.31.15:4698 , piece_id=KJZ2GOHKXXWREHUAIMSJ6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.943383035 (56h9m51.494853496s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:14:19.226465189 (56h10m17.77793565s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection Nov 11 16:14:19.226675380 (56h10m17.778145851s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection Nov 11 16:14:19.226719734 (56h10m17.778190195s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection Nov 11 16:14:19.226939854 (56h10m17.778410325s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: read tcp 188.166.225.170:46848->145.79.227.27:4689: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.226954371 (56h10m17.778424832s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection Nov 11 16:14:19.227194228 (56h10m17.778664699s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:47846->145.79.227.32:4834: use of closed network connection; context deadline exceeded Nov 11 16:14:19.227398437 (56h10m17.778868898s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:46890->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.228143486 (56h10m17.779614047s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection Nov 11 16:14:19.228201325 (56h10m17.779671797s) [seg_pos={0 0}, upload=49489]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 11 16:14:19.496444177 (56h10m18.047914638s) [upload=49489]: Done uploading segments... Nov 11 16:14:19.496495415 (56h10m18.047965877s) [upload=49489]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.903190013s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 56.392535ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/79b21962f2d8e182 FAIL (2025-11-11 16:22:35 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:55382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 56.423043ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.022880126s trace: https://jaeger.dev.storj.io/trace/44d8b784072367e4 FAIL (2025-11-11 16:22:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== iece... Nov 11 16:08:48.413266786 (56h4m46.964737247s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.413968803 (56h4m46.965439264s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414381078 (56h4m46.965851539s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414796339 (56h4m46.9662668s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415222971 (56h4m46.966693432s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415704538 (56h4m46.967174989s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416081355 (56h4m46.967551816s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416488901 (56h4m46.967959362s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416811835 (56h4m46.968282287s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417126284 (56h4m46.968596735s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417504084 (56h4m46.968974535s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.418850267 (56h4m46.970320719s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.419254357 (56h4m46.970724818s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.421629780 (56h4m46.973100241s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.423823017 (56h4m46.975293478s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.426626956 (56h4m46.978097417s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.428526082 (56h4m46.979996543s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:49.743855894 (56h4m48.295326345s) [upload=49413]: Next returned nil segment Nov 11 16:08:49.743869990 (56h4m48.295340451s) [upload=49413]: Waiting for error group managing segments... Nov 11 16:08:50.083885022 (56h4m48.635355483s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.099619755 (56h4m48.651090216s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.185889601 (56h4m48.737360062s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.210908903 (56h4m48.762379354s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.233314608 (56h4m48.784785069s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.385166156 (56h4m48.936636617s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.412131434 (56h4m48.963601895s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.421729551 (56h4m48.973200012s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.430205912 (56h4m48.981676373s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.455736998 (56h4m49.007207459s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.497672944 (56h4m49.049143395s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.511232905 (56h4m49.062703366s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.569351094 (56h4m49.120821555s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:09:47.409293414 (56h5m45.960763875s) [noise=true , address=145.79.227.17:4994 , piece_id=ALC7PHRXP3EPN37WJZ5K4CF7SENX2T2T4GZZVEGCKO6EUXHL4GYA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:13nneaPpsRT7mVEc6cDt7GiZSMTU68Nnk1rr9Lbn5ycZkmQGrB, address:145.79.227.17:4994): protocol: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection Nov 11 16:09:47.409343289 (56h5m45.96081375s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection Nov 11 16:09:47.409602432 (56h5m45.961072903s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T, address:145.79.227.27:4707): protocol: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection Nov 11 16:09:47.409644141 (56h5m45.961114602s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection Nov 11 16:09:47.409737079 (56h5m45.96120753s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:12NBsd33JtAwPdcbD8vBzZUYpjiqZpRzs8y2QrYJFE245pd8T5X, address:145.79.227.44:4228): protocol: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection Nov 11 16:09:47.409766335 (56h5m45.961236786s) [seg_pos={0 0}, upload=49413]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Nov 11 16:09:47.658593534 (56h5m46.210063996s) [upload=49413]: Done uploading segments... Nov 11 16:09:47.658685831 (56h5m46.210156282s) [upload=49413]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.022904114s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 70.941448ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/79b21962f2d8e182 FAIL (2025-11-11 16:21:51 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:55382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 70.959482ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................XX.....X...........] duration: 7.177330318s trace: https://jaeger.dev.storj.io/trace/6ba329aed526607a FAIL (2025-11-11 16:21:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== J6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.204239805 (56h9m48.755710266s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205024630 (56h9m48.756495091s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205617369 (56h9m48.75708783s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.206216129 (56h9m48.75768659s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.207869137 (56h9m48.759339598s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.209417975 (56h9m48.760888436s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.210891231 (56h9m48.762361693s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.212264837 (56h9m48.763735288s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.213197944 (56h9m48.764668405s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.217083202 (56h9m48.768553663s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.218920661 (56h9m48.770391122s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.220461635 (56h9m48.771932086s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.221453104 (56h9m48.772923565s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.222919085 (56h9m48.774389576s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.224462674 (56h9m48.775933135s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:51.977601348 (56h9m50.52907181s) [upload=49489]: Next returned nil segment Nov 11 16:13:51.977618881 (56h9m50.529089342s) [upload=49489]: Waiting for error group managing segments... Nov 11 16:13:52.318041740 (56h9m50.869512201s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.319228510 (56h9m50.870699001s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.330974096 (56h9m50.882444547s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.366779437 (56h9m50.918249898s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.403498277 (56h9m50.954968738s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.410852602 (56h9m50.962323093s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.441652722 (56h9m50.993123183s) [noise=true , address=77.90.31.15:4698 , piece_id=KJZ2GOHKXXWREHUAIMSJ6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.943383035 (56h9m51.494853496s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:14:19.226465189 (56h10m17.77793565s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection Nov 11 16:14:19.226675380 (56h10m17.778145851s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection Nov 11 16:14:19.226719734 (56h10m17.778190195s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection Nov 11 16:14:19.226939854 (56h10m17.778410325s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: read tcp 188.166.225.170:46848->145.79.227.27:4689: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.226954371 (56h10m17.778424832s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection Nov 11 16:14:19.227194228 (56h10m17.778664699s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:47846->145.79.227.32:4834: use of closed network connection; context deadline exceeded Nov 11 16:14:19.227398437 (56h10m17.778868898s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:46890->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.228143486 (56h10m17.779614047s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection Nov 11 16:14:19.228201325 (56h10m17.779671797s) [seg_pos={0 0}, upload=49489]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 11 16:14:19.496444177 (56h10m18.047914638s) [upload=49489]: Done uploading segments... Nov 11 16:14:19.496495415 (56h10m18.047965877s) [upload=49489]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 7.177361719s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 537.613433ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4d1e117d2094896e FAIL (2025-11-11 16:21:37 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:33304->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 537.642579ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.979802049s trace: https://jaeger.dev.storj.io/trace/44d8b784072367e4 FAIL (2025-11-11 16:21:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== iece... Nov 11 16:08:48.413266786 (56h4m46.964737247s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.413968803 (56h4m46.965439264s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414381078 (56h4m46.965851539s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414796339 (56h4m46.9662668s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415222971 (56h4m46.966693432s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415704538 (56h4m46.967174989s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416081355 (56h4m46.967551816s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416488901 (56h4m46.967959362s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416811835 (56h4m46.968282287s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417126284 (56h4m46.968596735s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417504084 (56h4m46.968974535s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.418850267 (56h4m46.970320719s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.419254357 (56h4m46.970724818s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.421629780 (56h4m46.973100241s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.423823017 (56h4m46.975293478s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.426626956 (56h4m46.978097417s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.428526082 (56h4m46.979996543s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:49.743855894 (56h4m48.295326345s) [upload=49413]: Next returned nil segment Nov 11 16:08:49.743869990 (56h4m48.295340451s) [upload=49413]: Waiting for error group managing segments... Nov 11 16:08:50.083885022 (56h4m48.635355483s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.099619755 (56h4m48.651090216s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.185889601 (56h4m48.737360062s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.210908903 (56h4m48.762379354s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.233314608 (56h4m48.784785069s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.385166156 (56h4m48.936636617s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.412131434 (56h4m48.963601895s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.421729551 (56h4m48.973200012s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.430205912 (56h4m48.981676373s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.455736998 (56h4m49.007207459s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.497672944 (56h4m49.049143395s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.511232905 (56h4m49.062703366s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.569351094 (56h4m49.120821555s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:09:47.409293414 (56h5m45.960763875s) [noise=true , address=145.79.227.17:4994 , piece_id=ALC7PHRXP3EPN37WJZ5K4CF7SENX2T2T4GZZVEGCKO6EUXHL4GYA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:13nneaPpsRT7mVEc6cDt7GiZSMTU68Nnk1rr9Lbn5ycZkmQGrB, address:145.79.227.17:4994): protocol: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection Nov 11 16:09:47.409343289 (56h5m45.96081375s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection Nov 11 16:09:47.409602432 (56h5m45.961072903s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T, address:145.79.227.27:4707): protocol: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection Nov 11 16:09:47.409644141 (56h5m45.961114602s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection Nov 11 16:09:47.409737079 (56h5m45.96120753s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:12NBsd33JtAwPdcbD8vBzZUYpjiqZpRzs8y2QrYJFE245pd8T5X, address:145.79.227.44:4228): protocol: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection Nov 11 16:09:47.409766335 (56h5m45.961236786s) [seg_pos={0 0}, upload=49413]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Nov 11 16:09:47.658593534 (56h5m46.210063996s) [upload=49413]: Done uploading segments... Nov 11 16:09:47.658685831 (56h5m46.210156282s) [upload=49413]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.979833449s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 324.068195ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4d1e117d2094896e FAIL (2025-11-11 16:21:08 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:33304->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 324.100949ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................................XX.....X..........] duration: 7.102858725s trace: https://jaeger.dev.storj.io/trace/6ba329aed526607a FAIL (2025-11-11 16:21:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== J6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.204239805 (56h9m48.755710266s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205024630 (56h9m48.756495091s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205617369 (56h9m48.75708783s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.206216129 (56h9m48.75768659s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.207869137 (56h9m48.759339598s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.209417975 (56h9m48.760888436s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.210891231 (56h9m48.762361693s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.212264837 (56h9m48.763735288s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.213197944 (56h9m48.764668405s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.217083202 (56h9m48.768553663s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.218920661 (56h9m48.770391122s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.220461635 (56h9m48.771932086s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.221453104 (56h9m48.772923565s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.222919085 (56h9m48.774389576s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.224462674 (56h9m48.775933135s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:51.977601348 (56h9m50.52907181s) [upload=49489]: Next returned nil segment Nov 11 16:13:51.977618881 (56h9m50.529089342s) [upload=49489]: Waiting for error group managing segments... Nov 11 16:13:52.318041740 (56h9m50.869512201s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.319228510 (56h9m50.870699001s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.330974096 (56h9m50.882444547s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.366779437 (56h9m50.918249898s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.403498277 (56h9m50.954968738s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.410852602 (56h9m50.962323093s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.441652722 (56h9m50.993123183s) [noise=true , address=77.90.31.15:4698 , piece_id=KJZ2GOHKXXWREHUAIMSJ6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.943383035 (56h9m51.494853496s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:14:19.226465189 (56h10m17.77793565s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection Nov 11 16:14:19.226675380 (56h10m17.778145851s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection Nov 11 16:14:19.226719734 (56h10m17.778190195s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection Nov 11 16:14:19.226939854 (56h10m17.778410325s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: read tcp 188.166.225.170:46848->145.79.227.27:4689: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.226954371 (56h10m17.778424832s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection Nov 11 16:14:19.227194228 (56h10m17.778664699s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:47846->145.79.227.32:4834: use of closed network connection; context deadline exceeded Nov 11 16:14:19.227398437 (56h10m17.778868898s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:46890->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.228143486 (56h10m17.779614047s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection Nov 11 16:14:19.228201325 (56h10m17.779671797s) [seg_pos={0 0}, upload=49489]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 11 16:14:19.496444177 (56h10m18.047914638s) [upload=49489]: Done uploading segments... Nov 11 16:14:19.496495415 (56h10m18.047965877s) [upload=49489]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 7.102881249s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 67.375938ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/79b21962f2d8e182 FAIL (2025-11-11 16:20:43 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:55382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 67.407408ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.734923147s trace: https://jaeger.dev.storj.io/trace/44d8b784072367e4 FAIL (2025-11-11 16:20:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== iece... Nov 11 16:08:48.413266786 (56h4m46.964737247s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.413968803 (56h4m46.965439264s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414381078 (56h4m46.965851539s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414796339 (56h4m46.9662668s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415222971 (56h4m46.966693432s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415704538 (56h4m46.967174989s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416081355 (56h4m46.967551816s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416488901 (56h4m46.967959362s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416811835 (56h4m46.968282287s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417126284 (56h4m46.968596735s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417504084 (56h4m46.968974535s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.418850267 (56h4m46.970320719s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.419254357 (56h4m46.970724818s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.421629780 (56h4m46.973100241s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.423823017 (56h4m46.975293478s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.426626956 (56h4m46.978097417s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.428526082 (56h4m46.979996543s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:49.743855894 (56h4m48.295326345s) [upload=49413]: Next returned nil segment Nov 11 16:08:49.743869990 (56h4m48.295340451s) [upload=49413]: Waiting for error group managing segments... Nov 11 16:08:50.083885022 (56h4m48.635355483s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.099619755 (56h4m48.651090216s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.185889601 (56h4m48.737360062s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.210908903 (56h4m48.762379354s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.233314608 (56h4m48.784785069s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.385166156 (56h4m48.936636617s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.412131434 (56h4m48.963601895s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.421729551 (56h4m48.973200012s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.430205912 (56h4m48.981676373s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.455736998 (56h4m49.007207459s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.497672944 (56h4m49.049143395s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.511232905 (56h4m49.062703366s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.569351094 (56h4m49.120821555s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:09:47.409293414 (56h5m45.960763875s) [noise=true , address=145.79.227.17:4994 , piece_id=ALC7PHRXP3EPN37WJZ5K4CF7SENX2T2T4GZZVEGCKO6EUXHL4GYA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:13nneaPpsRT7mVEc6cDt7GiZSMTU68Nnk1rr9Lbn5ycZkmQGrB, address:145.79.227.17:4994): protocol: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection Nov 11 16:09:47.409343289 (56h5m45.96081375s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection Nov 11 16:09:47.409602432 (56h5m45.961072903s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T, address:145.79.227.27:4707): protocol: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection Nov 11 16:09:47.409644141 (56h5m45.961114602s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection Nov 11 16:09:47.409737079 (56h5m45.96120753s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:12NBsd33JtAwPdcbD8vBzZUYpjiqZpRzs8y2QrYJFE245pd8T5X, address:145.79.227.44:4228): protocol: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection Nov 11 16:09:47.409766335 (56h5m45.961236786s) [seg_pos={0 0}, upload=49413]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Nov 11 16:09:47.658593534 (56h5m46.210063996s) [upload=49413]: Done uploading segments... Nov 11 16:09:47.658685831 (56h5m46.210156282s) [upload=49413]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.73495061s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [X..........................................XX.....X.........] duration: 6.745475542s trace: https://jaeger.dev.storj.io/trace/6ba329aed526607a FAIL (2025-11-11 16:20:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== J6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.204239805 (56h9m48.755710266s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205024630 (56h9m48.756495091s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205617369 (56h9m48.75708783s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.206216129 (56h9m48.75768659s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.207869137 (56h9m48.759339598s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.209417975 (56h9m48.760888436s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.210891231 (56h9m48.762361693s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.212264837 (56h9m48.763735288s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.213197944 (56h9m48.764668405s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.217083202 (56h9m48.768553663s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.218920661 (56h9m48.770391122s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.220461635 (56h9m48.771932086s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.221453104 (56h9m48.772923565s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.222919085 (56h9m48.774389576s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.224462674 (56h9m48.775933135s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:51.977601348 (56h9m50.52907181s) [upload=49489]: Next returned nil segment Nov 11 16:13:51.977618881 (56h9m50.529089342s) [upload=49489]: Waiting for error group managing segments... Nov 11 16:13:52.318041740 (56h9m50.869512201s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.319228510 (56h9m50.870699001s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.330974096 (56h9m50.882444547s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.366779437 (56h9m50.918249898s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.403498277 (56h9m50.954968738s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.410852602 (56h9m50.962323093s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.441652722 (56h9m50.993123183s) [noise=true , address=77.90.31.15:4698 , piece_id=KJZ2GOHKXXWREHUAIMSJ6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.943383035 (56h9m51.494853496s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:14:19.226465189 (56h10m17.77793565s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection Nov 11 16:14:19.226675380 (56h10m17.778145851s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection Nov 11 16:14:19.226719734 (56h10m17.778190195s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection Nov 11 16:14:19.226939854 (56h10m17.778410325s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: read tcp 188.166.225.170:46848->145.79.227.27:4689: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.226954371 (56h10m17.778424832s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection Nov 11 16:14:19.227194228 (56h10m17.778664699s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:47846->145.79.227.32:4834: use of closed network connection; context deadline exceeded Nov 11 16:14:19.227398437 (56h10m17.778868898s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:46890->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.228143486 (56h10m17.779614047s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection Nov 11 16:14:19.228201325 (56h10m17.779671797s) [seg_pos={0 0}, upload=49489]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 11 16:14:19.496444177 (56h10m18.047914638s) [upload=49489]: Done uploading segments... Nov 11 16:14:19.496495415 (56h10m18.047965877s) [upload=49489]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.745503024s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 551.548858ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4d1e117d2094896e FAIL (2025-11-11 16:20:18 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:33304->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 551.57062ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 82.520497ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/79b21962f2d8e182 FAIL (2025-11-11 16:20:01 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:55382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 82.555483ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 320.64201ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4d1e117d2094896e FAIL (2025-11-11 16:19:52 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:33304->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 320.668861ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 59.4788ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/79b21962f2d8e182 FAIL (2025-11-11 16:19:33 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:55382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 59.593759ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.665834479s trace: https://jaeger.dev.storj.io/trace/44d8b784072367e4 FAIL (2025-11-11 16:19:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== iece... Nov 11 16:08:48.413266786 (56h4m46.964737247s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.413968803 (56h4m46.965439264s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414381078 (56h4m46.965851539s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414796339 (56h4m46.9662668s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415222971 (56h4m46.966693432s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415704538 (56h4m46.967174989s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416081355 (56h4m46.967551816s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416488901 (56h4m46.967959362s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416811835 (56h4m46.968282287s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417126284 (56h4m46.968596735s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417504084 (56h4m46.968974535s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.418850267 (56h4m46.970320719s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.419254357 (56h4m46.970724818s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.421629780 (56h4m46.973100241s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.423823017 (56h4m46.975293478s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.426626956 (56h4m46.978097417s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.428526082 (56h4m46.979996543s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:49.743855894 (56h4m48.295326345s) [upload=49413]: Next returned nil segment Nov 11 16:08:49.743869990 (56h4m48.295340451s) [upload=49413]: Waiting for error group managing segments... Nov 11 16:08:50.083885022 (56h4m48.635355483s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.099619755 (56h4m48.651090216s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.185889601 (56h4m48.737360062s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.210908903 (56h4m48.762379354s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.233314608 (56h4m48.784785069s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.385166156 (56h4m48.936636617s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.412131434 (56h4m48.963601895s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.421729551 (56h4m48.973200012s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.430205912 (56h4m48.981676373s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.455736998 (56h4m49.007207459s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.497672944 (56h4m49.049143395s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.511232905 (56h4m49.062703366s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.569351094 (56h4m49.120821555s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:09:47.409293414 (56h5m45.960763875s) [noise=true , address=145.79.227.17:4994 , piece_id=ALC7PHRXP3EPN37WJZ5K4CF7SENX2T2T4GZZVEGCKO6EUXHL4GYA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:13nneaPpsRT7mVEc6cDt7GiZSMTU68Nnk1rr9Lbn5ycZkmQGrB, address:145.79.227.17:4994): protocol: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection Nov 11 16:09:47.409343289 (56h5m45.96081375s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection Nov 11 16:09:47.409602432 (56h5m45.961072903s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T, address:145.79.227.27:4707): protocol: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection Nov 11 16:09:47.409644141 (56h5m45.961114602s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection Nov 11 16:09:47.409737079 (56h5m45.96120753s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:12NBsd33JtAwPdcbD8vBzZUYpjiqZpRzs8y2QrYJFE245pd8T5X, address:145.79.227.44:4228): protocol: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection Nov 11 16:09:47.409766335 (56h5m45.961236786s) [seg_pos={0 0}, upload=49413]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Nov 11 16:09:47.658593534 (56h5m46.210063996s) [upload=49413]: Done uploading segments... Nov 11 16:09:47.658685831 (56h5m46.210156282s) [upload=49413]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.665859247s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X..........................................XX.....X........] duration: 7.414453578s trace: https://jaeger.dev.storj.io/trace/6ba329aed526607a FAIL (2025-11-11 16:19:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== J6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.204239805 (56h9m48.755710266s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205024630 (56h9m48.756495091s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205617369 (56h9m48.75708783s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.206216129 (56h9m48.75768659s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.207869137 (56h9m48.759339598s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.209417975 (56h9m48.760888436s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.210891231 (56h9m48.762361693s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.212264837 (56h9m48.763735288s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.213197944 (56h9m48.764668405s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.217083202 (56h9m48.768553663s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.218920661 (56h9m48.770391122s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.220461635 (56h9m48.771932086s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.221453104 (56h9m48.772923565s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.222919085 (56h9m48.774389576s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.224462674 (56h9m48.775933135s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:51.977601348 (56h9m50.52907181s) [upload=49489]: Next returned nil segment Nov 11 16:13:51.977618881 (56h9m50.529089342s) [upload=49489]: Waiting for error group managing segments... Nov 11 16:13:52.318041740 (56h9m50.869512201s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.319228510 (56h9m50.870699001s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.330974096 (56h9m50.882444547s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.366779437 (56h9m50.918249898s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.403498277 (56h9m50.954968738s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.410852602 (56h9m50.962323093s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.441652722 (56h9m50.993123183s) [noise=true , address=77.90.31.15:4698 , piece_id=KJZ2GOHKXXWREHUAIMSJ6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.943383035 (56h9m51.494853496s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:14:19.226465189 (56h10m17.77793565s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection Nov 11 16:14:19.226675380 (56h10m17.778145851s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection Nov 11 16:14:19.226719734 (56h10m17.778190195s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection Nov 11 16:14:19.226939854 (56h10m17.778410325s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: read tcp 188.166.225.170:46848->145.79.227.27:4689: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.226954371 (56h10m17.778424832s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection Nov 11 16:14:19.227194228 (56h10m17.778664699s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:47846->145.79.227.32:4834: use of closed network connection; context deadline exceeded Nov 11 16:14:19.227398437 (56h10m17.778868898s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:46890->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.228143486 (56h10m17.779614047s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection Nov 11 16:14:19.228201325 (56h10m17.779671797s) [seg_pos={0 0}, upload=49489]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 11 16:14:19.496444177 (56h10m18.047914638s) [upload=49489]: Done uploading segments... Nov 11 16:14:19.496495415 (56h10m18.047965877s) [upload=49489]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 7.414480449s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 334.209628ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4d1e117d2094896e FAIL (2025-11-11 16:19:03 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:33304->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 334.2892ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 336.613246ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4d1e117d2094896e FAIL (2025-11-11 16:18:45 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:33304->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 336.641429ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.104807518s trace: https://jaeger.dev.storj.io/trace/44d8b784072367e4 FAIL (2025-11-11 16:18:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== iece... Nov 11 16:08:48.413266786 (56h4m46.964737247s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.413968803 (56h4m46.965439264s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414381078 (56h4m46.965851539s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414796339 (56h4m46.9662668s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415222971 (56h4m46.966693432s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415704538 (56h4m46.967174989s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416081355 (56h4m46.967551816s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416488901 (56h4m46.967959362s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416811835 (56h4m46.968282287s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417126284 (56h4m46.968596735s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417504084 (56h4m46.968974535s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.418850267 (56h4m46.970320719s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.419254357 (56h4m46.970724818s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.421629780 (56h4m46.973100241s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.423823017 (56h4m46.975293478s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.426626956 (56h4m46.978097417s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.428526082 (56h4m46.979996543s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:49.743855894 (56h4m48.295326345s) [upload=49413]: Next returned nil segment Nov 11 16:08:49.743869990 (56h4m48.295340451s) [upload=49413]: Waiting for error group managing segments... Nov 11 16:08:50.083885022 (56h4m48.635355483s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.099619755 (56h4m48.651090216s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.185889601 (56h4m48.737360062s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.210908903 (56h4m48.762379354s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.233314608 (56h4m48.784785069s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.385166156 (56h4m48.936636617s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.412131434 (56h4m48.963601895s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.421729551 (56h4m48.973200012s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.430205912 (56h4m48.981676373s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.455736998 (56h4m49.007207459s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.497672944 (56h4m49.049143395s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.511232905 (56h4m49.062703366s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.569351094 (56h4m49.120821555s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:09:47.409293414 (56h5m45.960763875s) [noise=true , address=145.79.227.17:4994 , piece_id=ALC7PHRXP3EPN37WJZ5K4CF7SENX2T2T4GZZVEGCKO6EUXHL4GYA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:13nneaPpsRT7mVEc6cDt7GiZSMTU68Nnk1rr9Lbn5ycZkmQGrB, address:145.79.227.17:4994): protocol: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection Nov 11 16:09:47.409343289 (56h5m45.96081375s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection Nov 11 16:09:47.409602432 (56h5m45.961072903s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T, address:145.79.227.27:4707): protocol: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection Nov 11 16:09:47.409644141 (56h5m45.961114602s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection Nov 11 16:09:47.409737079 (56h5m45.96120753s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:12NBsd33JtAwPdcbD8vBzZUYpjiqZpRzs8y2QrYJFE245pd8T5X, address:145.79.227.44:4228): protocol: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection Nov 11 16:09:47.409766335 (56h5m45.961236786s) [seg_pos={0 0}, upload=49413]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Nov 11 16:09:47.658593534 (56h5m46.210063996s) [upload=49413]: Done uploading segments... Nov 11 16:09:47.658685831 (56h5m46.210156282s) [upload=49413]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.104832055s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 77.914526ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/79b21962f2d8e182 FAIL (2025-11-11 16:18:31 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:55382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 77.944123ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X..........................................XX.....X.......] duration: 8.083480726s trace: https://jaeger.dev.storj.io/trace/6ba329aed526607a FAIL (2025-11-11 16:18:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== J6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.204239805 (56h9m48.755710266s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205024630 (56h9m48.756495091s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205617369 (56h9m48.75708783s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.206216129 (56h9m48.75768659s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.207869137 (56h9m48.759339598s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.209417975 (56h9m48.760888436s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.210891231 (56h9m48.762361693s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.212264837 (56h9m48.763735288s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.213197944 (56h9m48.764668405s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.217083202 (56h9m48.768553663s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.218920661 (56h9m48.770391122s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.220461635 (56h9m48.771932086s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.221453104 (56h9m48.772923565s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.222919085 (56h9m48.774389576s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.224462674 (56h9m48.775933135s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:51.977601348 (56h9m50.52907181s) [upload=49489]: Next returned nil segment Nov 11 16:13:51.977618881 (56h9m50.529089342s) [upload=49489]: Waiting for error group managing segments... Nov 11 16:13:52.318041740 (56h9m50.869512201s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.319228510 (56h9m50.870699001s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.330974096 (56h9m50.882444547s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.366779437 (56h9m50.918249898s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.403498277 (56h9m50.954968738s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.410852602 (56h9m50.962323093s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.441652722 (56h9m50.993123183s) [noise=true , address=77.90.31.15:4698 , piece_id=KJZ2GOHKXXWREHUAIMSJ6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.943383035 (56h9m51.494853496s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:14:19.226465189 (56h10m17.77793565s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection Nov 11 16:14:19.226675380 (56h10m17.778145851s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection Nov 11 16:14:19.226719734 (56h10m17.778190195s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection Nov 11 16:14:19.226939854 (56h10m17.778410325s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: read tcp 188.166.225.170:46848->145.79.227.27:4689: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.226954371 (56h10m17.778424832s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection Nov 11 16:14:19.227194228 (56h10m17.778664699s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:47846->145.79.227.32:4834: use of closed network connection; context deadline exceeded Nov 11 16:14:19.227398437 (56h10m17.778868898s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:46890->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.228143486 (56h10m17.779614047s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection Nov 11 16:14:19.228201325 (56h10m17.779671797s) [seg_pos={0 0}, upload=49489]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 11 16:14:19.496444177 (56h10m18.047914638s) [upload=49489]: Done uploading segments... Nov 11 16:14:19.496495415 (56h10m18.047965877s) [upload=49489]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 8.083508379s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 83.850148ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/79b21962f2d8e182 FAIL (2025-11-11 16:18:12 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:55382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 83.884093ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...X..........................................XX.....X......] duration: 6.793188732s trace: https://jaeger.dev.storj.io/trace/6ba329aed526607a FAIL (2025-11-11 16:18:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== J6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.204239805 (56h9m48.755710266s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205024630 (56h9m48.756495091s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205617369 (56h9m48.75708783s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.206216129 (56h9m48.75768659s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.207869137 (56h9m48.759339598s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.209417975 (56h9m48.760888436s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.210891231 (56h9m48.762361693s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.212264837 (56h9m48.763735288s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.213197944 (56h9m48.764668405s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.217083202 (56h9m48.768553663s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.218920661 (56h9m48.770391122s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.220461635 (56h9m48.771932086s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.221453104 (56h9m48.772923565s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.222919085 (56h9m48.774389576s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.224462674 (56h9m48.775933135s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:51.977601348 (56h9m50.52907181s) [upload=49489]: Next returned nil segment Nov 11 16:13:51.977618881 (56h9m50.529089342s) [upload=49489]: Waiting for error group managing segments... Nov 11 16:13:52.318041740 (56h9m50.869512201s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.319228510 (56h9m50.870699001s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.330974096 (56h9m50.882444547s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.366779437 (56h9m50.918249898s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.403498277 (56h9m50.954968738s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.410852602 (56h9m50.962323093s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.441652722 (56h9m50.993123183s) [noise=true , address=77.90.31.15:4698 , piece_id=KJZ2GOHKXXWREHUAIMSJ6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.943383035 (56h9m51.494853496s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:14:19.226465189 (56h10m17.77793565s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection Nov 11 16:14:19.226675380 (56h10m17.778145851s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection Nov 11 16:14:19.226719734 (56h10m17.778190195s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection Nov 11 16:14:19.226939854 (56h10m17.778410325s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: read tcp 188.166.225.170:46848->145.79.227.27:4689: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.226954371 (56h10m17.778424832s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection Nov 11 16:14:19.227194228 (56h10m17.778664699s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:47846->145.79.227.32:4834: use of closed network connection; context deadline exceeded Nov 11 16:14:19.227398437 (56h10m17.778868898s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:46890->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.228143486 (56h10m17.779614047s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection Nov 11 16:14:19.228201325 (56h10m17.779671797s) [seg_pos={0 0}, upload=49489]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 11 16:14:19.496444177 (56h10m18.047914638s) [upload=49489]: Done uploading segments... Nov 11 16:14:19.496495415 (56h10m18.047965877s) [upload=49489]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.793221275s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 355.733694ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4d1e117d2094896e FAIL (2025-11-11 16:17:45 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:33304->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 355.773339ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.26891912s trace: https://jaeger.dev.storj.io/trace/44d8b784072367e4 FAIL (2025-11-11 16:17:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== iece... Nov 11 16:08:48.413266786 (56h4m46.964737247s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.413968803 (56h4m46.965439264s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414381078 (56h4m46.965851539s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414796339 (56h4m46.9662668s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415222971 (56h4m46.966693432s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415704538 (56h4m46.967174989s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416081355 (56h4m46.967551816s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416488901 (56h4m46.967959362s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416811835 (56h4m46.968282287s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417126284 (56h4m46.968596735s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417504084 (56h4m46.968974535s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.418850267 (56h4m46.970320719s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.419254357 (56h4m46.970724818s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.421629780 (56h4m46.973100241s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.423823017 (56h4m46.975293478s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.426626956 (56h4m46.978097417s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.428526082 (56h4m46.979996543s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:49.743855894 (56h4m48.295326345s) [upload=49413]: Next returned nil segment Nov 11 16:08:49.743869990 (56h4m48.295340451s) [upload=49413]: Waiting for error group managing segments... Nov 11 16:08:50.083885022 (56h4m48.635355483s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.099619755 (56h4m48.651090216s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.185889601 (56h4m48.737360062s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.210908903 (56h4m48.762379354s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.233314608 (56h4m48.784785069s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.385166156 (56h4m48.936636617s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.412131434 (56h4m48.963601895s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.421729551 (56h4m48.973200012s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.430205912 (56h4m48.981676373s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.455736998 (56h4m49.007207459s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.497672944 (56h4m49.049143395s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.511232905 (56h4m49.062703366s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.569351094 (56h4m49.120821555s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:09:47.409293414 (56h5m45.960763875s) [noise=true , address=145.79.227.17:4994 , piece_id=ALC7PHRXP3EPN37WJZ5K4CF7SENX2T2T4GZZVEGCKO6EUXHL4GYA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:13nneaPpsRT7mVEc6cDt7GiZSMTU68Nnk1rr9Lbn5ycZkmQGrB, address:145.79.227.17:4994): protocol: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection Nov 11 16:09:47.409343289 (56h5m45.96081375s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection Nov 11 16:09:47.409602432 (56h5m45.961072903s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T, address:145.79.227.27:4707): protocol: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection Nov 11 16:09:47.409644141 (56h5m45.961114602s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection Nov 11 16:09:47.409737079 (56h5m45.96120753s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:12NBsd33JtAwPdcbD8vBzZUYpjiqZpRzs8y2QrYJFE245pd8T5X, address:145.79.227.44:4228): protocol: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection Nov 11 16:09:47.409766335 (56h5m45.961236786s) [seg_pos={0 0}, upload=49413]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Nov 11 16:09:47.658593534 (56h5m46.210063996s) [upload=49413]: Done uploading segments... Nov 11 16:09:47.658685831 (56h5m46.210156282s) [upload=49413]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.268948056s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 342.158774ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4d1e117d2094896e FAIL (2025-11-11 16:17:14 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:33304->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 342.263613ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....X..........................................XX.....X.....] duration: 6.641923107s trace: https://jaeger.dev.storj.io/trace/6ba329aed526607a FAIL (2025-11-11 16:17:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== J6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.204239805 (56h9m48.755710266s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205024630 (56h9m48.756495091s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205617369 (56h9m48.75708783s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.206216129 (56h9m48.75768659s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.207869137 (56h9m48.759339598s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.209417975 (56h9m48.760888436s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.210891231 (56h9m48.762361693s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.212264837 (56h9m48.763735288s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.213197944 (56h9m48.764668405s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.217083202 (56h9m48.768553663s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.218920661 (56h9m48.770391122s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.220461635 (56h9m48.771932086s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.221453104 (56h9m48.772923565s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.222919085 (56h9m48.774389576s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.224462674 (56h9m48.775933135s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:51.977601348 (56h9m50.52907181s) [upload=49489]: Next returned nil segment Nov 11 16:13:51.977618881 (56h9m50.529089342s) [upload=49489]: Waiting for error group managing segments... Nov 11 16:13:52.318041740 (56h9m50.869512201s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.319228510 (56h9m50.870699001s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.330974096 (56h9m50.882444547s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.366779437 (56h9m50.918249898s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.403498277 (56h9m50.954968738s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.410852602 (56h9m50.962323093s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.441652722 (56h9m50.993123183s) [noise=true , address=77.90.31.15:4698 , piece_id=KJZ2GOHKXXWREHUAIMSJ6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.943383035 (56h9m51.494853496s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:14:19.226465189 (56h10m17.77793565s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection Nov 11 16:14:19.226675380 (56h10m17.778145851s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection Nov 11 16:14:19.226719734 (56h10m17.778190195s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection Nov 11 16:14:19.226939854 (56h10m17.778410325s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: read tcp 188.166.225.170:46848->145.79.227.27:4689: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.226954371 (56h10m17.778424832s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection Nov 11 16:14:19.227194228 (56h10m17.778664699s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:47846->145.79.227.32:4834: use of closed network connection; context deadline exceeded Nov 11 16:14:19.227398437 (56h10m17.778868898s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:46890->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.228143486 (56h10m17.779614047s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection Nov 11 16:14:19.228201325 (56h10m17.779671797s) [seg_pos={0 0}, upload=49489]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 11 16:14:19.496444177 (56h10m18.047914638s) [upload=49489]: Done uploading segments... Nov 11 16:14:19.496495415 (56h10m18.047965877s) [upload=49489]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.641955269s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 54.778581ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/79b21962f2d8e182 FAIL (2025-11-11 16:17:08 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:55382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 54.797908ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.03850238s trace: https://jaeger.dev.storj.io/trace/44d8b784072367e4 FAIL (2025-11-11 16:16:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== iece... Nov 11 16:08:48.413266786 (56h4m46.964737247s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.413968803 (56h4m46.965439264s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414381078 (56h4m46.965851539s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414796339 (56h4m46.9662668s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415222971 (56h4m46.966693432s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415704538 (56h4m46.967174989s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416081355 (56h4m46.967551816s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416488901 (56h4m46.967959362s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416811835 (56h4m46.968282287s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417126284 (56h4m46.968596735s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417504084 (56h4m46.968974535s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.418850267 (56h4m46.970320719s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.419254357 (56h4m46.970724818s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.421629780 (56h4m46.973100241s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.423823017 (56h4m46.975293478s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.426626956 (56h4m46.978097417s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.428526082 (56h4m46.979996543s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:49.743855894 (56h4m48.295326345s) [upload=49413]: Next returned nil segment Nov 11 16:08:49.743869990 (56h4m48.295340451s) [upload=49413]: Waiting for error group managing segments... Nov 11 16:08:50.083885022 (56h4m48.635355483s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.099619755 (56h4m48.651090216s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.185889601 (56h4m48.737360062s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.210908903 (56h4m48.762379354s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.233314608 (56h4m48.784785069s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.385166156 (56h4m48.936636617s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.412131434 (56h4m48.963601895s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.421729551 (56h4m48.973200012s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.430205912 (56h4m48.981676373s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.455736998 (56h4m49.007207459s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.497672944 (56h4m49.049143395s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.511232905 (56h4m49.062703366s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.569351094 (56h4m49.120821555s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:09:47.409293414 (56h5m45.960763875s) [noise=true , address=145.79.227.17:4994 , piece_id=ALC7PHRXP3EPN37WJZ5K4CF7SENX2T2T4GZZVEGCKO6EUXHL4GYA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:13nneaPpsRT7mVEc6cDt7GiZSMTU68Nnk1rr9Lbn5ycZkmQGrB, address:145.79.227.17:4994): protocol: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection Nov 11 16:09:47.409343289 (56h5m45.96081375s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection Nov 11 16:09:47.409602432 (56h5m45.961072903s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T, address:145.79.227.27:4707): protocol: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection Nov 11 16:09:47.409644141 (56h5m45.961114602s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection Nov 11 16:09:47.409737079 (56h5m45.96120753s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:12NBsd33JtAwPdcbD8vBzZUYpjiqZpRzs8y2QrYJFE245pd8T5X, address:145.79.227.44:4228): protocol: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection Nov 11 16:09:47.409766335 (56h5m45.961236786s) [seg_pos={0 0}, upload=49413]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Nov 11 16:09:47.658593534 (56h5m46.210063996s) [upload=49413]: Done uploading segments... Nov 11 16:09:47.658685831 (56h5m46.210156282s) [upload=49413]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.038533879s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 53.685358ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/79b21962f2d8e182 FAIL (2025-11-11 16:16:52 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:55382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 53.767505ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 597.645316ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4d1e117d2094896e FAIL (2025-11-11 16:16:51 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:33304->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 597.682225ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....X..........................................XX.....X....] duration: 11.482054688s trace: https://jaeger.dev.storj.io/trace/6ba329aed526607a FAIL (2025-11-11 16:16:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== J6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.204239805 (56h9m48.755710266s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205024630 (56h9m48.756495091s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205617369 (56h9m48.75708783s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.206216129 (56h9m48.75768659s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.207869137 (56h9m48.759339598s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.209417975 (56h9m48.760888436s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.210891231 (56h9m48.762361693s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.212264837 (56h9m48.763735288s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.213197944 (56h9m48.764668405s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.217083202 (56h9m48.768553663s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.218920661 (56h9m48.770391122s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.220461635 (56h9m48.771932086s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.221453104 (56h9m48.772923565s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.222919085 (56h9m48.774389576s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.224462674 (56h9m48.775933135s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:51.977601348 (56h9m50.52907181s) [upload=49489]: Next returned nil segment Nov 11 16:13:51.977618881 (56h9m50.529089342s) [upload=49489]: Waiting for error group managing segments... Nov 11 16:13:52.318041740 (56h9m50.869512201s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.319228510 (56h9m50.870699001s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.330974096 (56h9m50.882444547s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.366779437 (56h9m50.918249898s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.403498277 (56h9m50.954968738s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.410852602 (56h9m50.962323093s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.441652722 (56h9m50.993123183s) [noise=true , address=77.90.31.15:4698 , piece_id=KJZ2GOHKXXWREHUAIMSJ6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.943383035 (56h9m51.494853496s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:14:19.226465189 (56h10m17.77793565s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection Nov 11 16:14:19.226675380 (56h10m17.778145851s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection Nov 11 16:14:19.226719734 (56h10m17.778190195s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection Nov 11 16:14:19.226939854 (56h10m17.778410325s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: read tcp 188.166.225.170:46848->145.79.227.27:4689: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.226954371 (56h10m17.778424832s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection Nov 11 16:14:19.227194228 (56h10m17.778664699s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:47846->145.79.227.32:4834: use of closed network connection; context deadline exceeded Nov 11 16:14:19.227398437 (56h10m17.778868898s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:46890->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.228143486 (56h10m17.779614047s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection Nov 11 16:14:19.228201325 (56h10m17.779671797s) [seg_pos={0 0}, upload=49489]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 11 16:14:19.496444177 (56h10m18.047914638s) [upload=49489]: Done uploading segments... Nov 11 16:14:19.496495415 (56h10m18.047965877s) [upload=49489]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 11.482083432s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 325.779024ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4d1e117d2094896e FAIL (2025-11-11 16:16:25 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:33304->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 325.807668ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......X..........................................XX.....X...] duration: 6.538918919s trace: https://jaeger.dev.storj.io/trace/6ba329aed526607a FAIL (2025-11-11 16:16:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== J6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.204239805 (56h9m48.755710266s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205024630 (56h9m48.756495091s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205617369 (56h9m48.75708783s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.206216129 (56h9m48.75768659s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.207869137 (56h9m48.759339598s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.209417975 (56h9m48.760888436s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.210891231 (56h9m48.762361693s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.212264837 (56h9m48.763735288s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.213197944 (56h9m48.764668405s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.217083202 (56h9m48.768553663s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.218920661 (56h9m48.770391122s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.220461635 (56h9m48.771932086s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.221453104 (56h9m48.772923565s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.222919085 (56h9m48.774389576s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.224462674 (56h9m48.775933135s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:51.977601348 (56h9m50.52907181s) [upload=49489]: Next returned nil segment Nov 11 16:13:51.977618881 (56h9m50.529089342s) [upload=49489]: Waiting for error group managing segments... Nov 11 16:13:52.318041740 (56h9m50.869512201s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.319228510 (56h9m50.870699001s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.330974096 (56h9m50.882444547s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.366779437 (56h9m50.918249898s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.403498277 (56h9m50.954968738s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.410852602 (56h9m50.962323093s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.441652722 (56h9m50.993123183s) [noise=true , address=77.90.31.15:4698 , piece_id=KJZ2GOHKXXWREHUAIMSJ6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.943383035 (56h9m51.494853496s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:14:19.226465189 (56h10m17.77793565s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection Nov 11 16:14:19.226675380 (56h10m17.778145851s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection Nov 11 16:14:19.226719734 (56h10m17.778190195s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection Nov 11 16:14:19.226939854 (56h10m17.778410325s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: read tcp 188.166.225.170:46848->145.79.227.27:4689: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.226954371 (56h10m17.778424832s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection Nov 11 16:14:19.227194228 (56h10m17.778664699s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:47846->145.79.227.32:4834: use of closed network connection; context deadline exceeded Nov 11 16:14:19.227398437 (56h10m17.778868898s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:46890->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.228143486 (56h10m17.779614047s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection Nov 11 16:14:19.228201325 (56h10m17.779671797s) [seg_pos={0 0}, upload=49489]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 11 16:14:19.496444177 (56h10m18.047914638s) [upload=49489]: Done uploading segments... Nov 11 16:14:19.496495415 (56h10m18.047965877s) [upload=49489]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.53894614s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.110727041s trace: https://jaeger.dev.storj.io/trace/44d8b784072367e4 FAIL (2025-11-11 16:16:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== iece... Nov 11 16:08:48.413266786 (56h4m46.964737247s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.413968803 (56h4m46.965439264s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414381078 (56h4m46.965851539s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414796339 (56h4m46.9662668s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415222971 (56h4m46.966693432s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415704538 (56h4m46.967174989s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416081355 (56h4m46.967551816s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416488901 (56h4m46.967959362s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416811835 (56h4m46.968282287s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417126284 (56h4m46.968596735s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417504084 (56h4m46.968974535s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.418850267 (56h4m46.970320719s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.419254357 (56h4m46.970724818s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.421629780 (56h4m46.973100241s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.423823017 (56h4m46.975293478s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.426626956 (56h4m46.978097417s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.428526082 (56h4m46.979996543s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:49.743855894 (56h4m48.295326345s) [upload=49413]: Next returned nil segment Nov 11 16:08:49.743869990 (56h4m48.295340451s) [upload=49413]: Waiting for error group managing segments... Nov 11 16:08:50.083885022 (56h4m48.635355483s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.099619755 (56h4m48.651090216s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.185889601 (56h4m48.737360062s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.210908903 (56h4m48.762379354s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.233314608 (56h4m48.784785069s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.385166156 (56h4m48.936636617s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.412131434 (56h4m48.963601895s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.421729551 (56h4m48.973200012s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.430205912 (56h4m48.981676373s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.455736998 (56h4m49.007207459s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.497672944 (56h4m49.049143395s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.511232905 (56h4m49.062703366s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.569351094 (56h4m49.120821555s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:09:47.409293414 (56h5m45.960763875s) [noise=true , address=145.79.227.17:4994 , piece_id=ALC7PHRXP3EPN37WJZ5K4CF7SENX2T2T4GZZVEGCKO6EUXHL4GYA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:13nneaPpsRT7mVEc6cDt7GiZSMTU68Nnk1rr9Lbn5ycZkmQGrB, address:145.79.227.17:4994): protocol: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection Nov 11 16:09:47.409343289 (56h5m45.96081375s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection Nov 11 16:09:47.409602432 (56h5m45.961072903s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T, address:145.79.227.27:4707): protocol: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection Nov 11 16:09:47.409644141 (56h5m45.961114602s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection Nov 11 16:09:47.409737079 (56h5m45.96120753s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:12NBsd33JtAwPdcbD8vBzZUYpjiqZpRzs8y2QrYJFE245pd8T5X, address:145.79.227.44:4228): protocol: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection Nov 11 16:09:47.409766335 (56h5m45.961236786s) [seg_pos={0 0}, upload=49413]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Nov 11 16:09:47.658593534 (56h5m46.210063996s) [upload=49413]: Done uploading segments... Nov 11 16:09:47.658685831 (56h5m46.210156282s) [upload=49413]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.110751556s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 490.001327ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4d1e117d2094896e FAIL (2025-11-11 16:16:04 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:33304->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 490.04949ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......X..........................................XX.....X..] duration: 7.25120788s trace: https://jaeger.dev.storj.io/trace/6ba329aed526607a FAIL (2025-11-11 16:15:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== J6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.204239805 (56h9m48.755710266s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205024630 (56h9m48.756495091s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205617369 (56h9m48.75708783s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.206216129 (56h9m48.75768659s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.207869137 (56h9m48.759339598s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.209417975 (56h9m48.760888436s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.210891231 (56h9m48.762361693s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.212264837 (56h9m48.763735288s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.213197944 (56h9m48.764668405s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.217083202 (56h9m48.768553663s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.218920661 (56h9m48.770391122s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.220461635 (56h9m48.771932086s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.221453104 (56h9m48.772923565s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.222919085 (56h9m48.774389576s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.224462674 (56h9m48.775933135s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:51.977601348 (56h9m50.52907181s) [upload=49489]: Next returned nil segment Nov 11 16:13:51.977618881 (56h9m50.529089342s) [upload=49489]: Waiting for error group managing segments... Nov 11 16:13:52.318041740 (56h9m50.869512201s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.319228510 (56h9m50.870699001s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.330974096 (56h9m50.882444547s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.366779437 (56h9m50.918249898s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.403498277 (56h9m50.954968738s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.410852602 (56h9m50.962323093s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.441652722 (56h9m50.993123183s) [noise=true , address=77.90.31.15:4698 , piece_id=KJZ2GOHKXXWREHUAIMSJ6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.943383035 (56h9m51.494853496s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:14:19.226465189 (56h10m17.77793565s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection Nov 11 16:14:19.226675380 (56h10m17.778145851s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection Nov 11 16:14:19.226719734 (56h10m17.778190195s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection Nov 11 16:14:19.226939854 (56h10m17.778410325s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: read tcp 188.166.225.170:46848->145.79.227.27:4689: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.226954371 (56h10m17.778424832s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection Nov 11 16:14:19.227194228 (56h10m17.778664699s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:47846->145.79.227.32:4834: use of closed network connection; context deadline exceeded Nov 11 16:14:19.227398437 (56h10m17.778868898s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:46890->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.228143486 (56h10m17.779614047s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection Nov 11 16:14:19.228201325 (56h10m17.779671797s) [seg_pos={0 0}, upload=49489]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 11 16:14:19.496444177 (56h10m18.047914638s) [upload=49489]: Done uploading segments... Nov 11 16:14:19.496495415 (56h10m18.047965877s) [upload=49489]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 7.251249491s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 95.582019ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/79b21962f2d8e182 FAIL (2025-11-11 16:15:37 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:55382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 95.632876ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.379089101s trace: https://jaeger.dev.storj.io/trace/44d8b784072367e4 FAIL (2025-11-11 16:15:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== iece... Nov 11 16:08:48.413266786 (56h4m46.964737247s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.413968803 (56h4m46.965439264s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414381078 (56h4m46.965851539s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414796339 (56h4m46.9662668s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415222971 (56h4m46.966693432s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415704538 (56h4m46.967174989s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416081355 (56h4m46.967551816s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416488901 (56h4m46.967959362s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416811835 (56h4m46.968282287s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417126284 (56h4m46.968596735s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417504084 (56h4m46.968974535s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.418850267 (56h4m46.970320719s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.419254357 (56h4m46.970724818s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.421629780 (56h4m46.973100241s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.423823017 (56h4m46.975293478s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.426626956 (56h4m46.978097417s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.428526082 (56h4m46.979996543s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:49.743855894 (56h4m48.295326345s) [upload=49413]: Next returned nil segment Nov 11 16:08:49.743869990 (56h4m48.295340451s) [upload=49413]: Waiting for error group managing segments... Nov 11 16:08:50.083885022 (56h4m48.635355483s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.099619755 (56h4m48.651090216s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.185889601 (56h4m48.737360062s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.210908903 (56h4m48.762379354s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.233314608 (56h4m48.784785069s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.385166156 (56h4m48.936636617s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.412131434 (56h4m48.963601895s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.421729551 (56h4m48.973200012s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.430205912 (56h4m48.981676373s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.455736998 (56h4m49.007207459s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.497672944 (56h4m49.049143395s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.511232905 (56h4m49.062703366s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.569351094 (56h4m49.120821555s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:09:47.409293414 (56h5m45.960763875s) [noise=true , address=145.79.227.17:4994 , piece_id=ALC7PHRXP3EPN37WJZ5K4CF7SENX2T2T4GZZVEGCKO6EUXHL4GYA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:13nneaPpsRT7mVEc6cDt7GiZSMTU68Nnk1rr9Lbn5ycZkmQGrB, address:145.79.227.17:4994): protocol: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection Nov 11 16:09:47.409343289 (56h5m45.96081375s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection Nov 11 16:09:47.409602432 (56h5m45.961072903s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T, address:145.79.227.27:4707): protocol: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection Nov 11 16:09:47.409644141 (56h5m45.961114602s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection Nov 11 16:09:47.409737079 (56h5m45.96120753s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:12NBsd33JtAwPdcbD8vBzZUYpjiqZpRzs8y2QrYJFE245pd8T5X, address:145.79.227.44:4228): protocol: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection Nov 11 16:09:47.409766335 (56h5m45.961236786s) [seg_pos={0 0}, upload=49413]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Nov 11 16:09:47.658593534 (56h5m46.210063996s) [upload=49413]: Done uploading segments... Nov 11 16:09:47.658685831 (56h5m46.210156282s) [upload=49413]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.379179243s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 345.959351ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4d1e117d2094896e FAIL (2025-11-11 16:15:15 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:33304->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 345.989308ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.923071717s trace: https://jaeger.dev.storj.io/trace/44d8b784072367e4 FAIL (2025-11-11 16:14:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== iece... Nov 11 16:08:48.413266786 (56h4m46.964737247s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.413968803 (56h4m46.965439264s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414381078 (56h4m46.965851539s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.414796339 (56h4m46.9662668s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415222971 (56h4m46.966693432s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.415704538 (56h4m46.967174989s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416081355 (56h4m46.967551816s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416488901 (56h4m46.967959362s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.416811835 (56h4m46.968282287s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417126284 (56h4m46.968596735s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.417504084 (56h4m46.968974535s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.418850267 (56h4m46.970320719s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.419254357 (56h4m46.970724818s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.421629780 (56h4m46.973100241s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.423823017 (56h4m46.975293478s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.426626956 (56h4m46.978097417s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:48.428526082 (56h4m46.979996543s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Uploading piece... Nov 11 16:08:49.743855894 (56h4m48.295326345s) [upload=49413]: Next returned nil segment Nov 11 16:08:49.743869990 (56h4m48.295340451s) [upload=49413]: Waiting for error group managing segments... Nov 11 16:08:50.083885022 (56h4m48.635355483s) [noise=true , address=64.49.13.169:4708 , piece_id=BGOKG2A22ONOW3C42ZDGJVLCFYWBDZCNDNWEJFH5CI7IWNCEEJHQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.099619755 (56h4m48.651090216s) [noise=true , address=64.49.13.154:4930 , piece_id=QJDOKUCE66TGMZBIH2H3X6VKIA6TEUY3FLWWHHOGKDFAG73E5APA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.185889601 (56h4m48.737360062s) [noise=true , address=5.231.73.3:4014 , piece_id=I6JOMX6V6RHVI56BNK5QEP7I66DXLMFPXBVLV7MDEHGI4DQYVPGA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.210908903 (56h4m48.762379354s) [noise=true , address=5.231.73.2:4019 , piece_id=RZ6A6LCAFTR5QEU4HRDH6VPZHJLLIYEEIKPYYSHPIYLKYWJTSGIQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.233314608 (56h4m48.784785069s) [noise=true , address=5.231.73.5:4029 , piece_id=BFGNXLGWWONZVAU2YN5M72C4GRVMLEY4UPMYONODCJU4SBOMPKCA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.385166156 (56h4m48.936636617s) [noise=true , address=45.91.116.8:4073 , piece_id=NKCUJYJCSSWAUYG7DG7ZHZK7MTD7NI5QWAEDP5OSUQ56NVE65DAA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.412131434 (56h4m48.963601895s) [noise=true , address=45.91.116.9:4289 , piece_id=Z5IN2YPT34E7OOXG5DUULXXJBNSFTBLKHUECCQ7B3V43BYNYYU3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.421729551 (56h4m48.973200012s) [noise=true , address=77.90.31.18:4808 , piece_id=WG7FSO6WEY2MZVTNZ4UPRZOT57ZZSBWADNHRHCO7S7CWIFIHA4OA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.430205912 (56h4m48.981676373s) [noise=true , address=77.90.31.14:4677 , piece_id=AT7AAAXAFYRVEWXPJLBTN6FHOLXRHSIP5UZVIBUJGR7ZKJWCCV3Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.455736998 (56h4m49.007207459s) [noise=true , address=77.90.31.11:4580 , piece_id=SUUPKQKDEP7FD5ZBPWCWMDDX4O5HNXYTDQCH3SF52PBRONBJX3BA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.497672944 (56h4m49.049143395s) [noise=true , address=77.90.31.17:4782 , piece_id=6ZE4O6KNM7BAPNEQGEUJ5AIPQT2MUS7B4HSPX6O4CHRZZ7RDVLTA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.511232905 (56h4m49.062703366s) [noise=true , address=77.90.31.13:4650 , piece_id=BBV45CNGTXMXMWYS4OMFLKOT5KOPH2GBB6NA57S47JW75AOQXVYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:08:50.569351094 (56h4m49.120821555s) [noise=true , address=77.90.31.19:4871 , piece_id=P7NSRE4J5GU56IKHETU65J32VUJZ6GCML3TLCFUUBZOLTOT7FNNQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: <nil> Nov 11 16:09:47.409293414 (56h5m45.960763875s) [noise=true , address=145.79.227.17:4994 , piece_id=ALC7PHRXP3EPN37WJZ5K4CF7SENX2T2T4GZZVEGCKO6EUXHL4GYA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:13nneaPpsRT7mVEc6cDt7GiZSMTU68Nnk1rr9Lbn5ycZkmQGrB, address:145.79.227.17:4994): protocol: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36002->145.79.227.17:4994: use of closed network connection Nov 11 16:09:47.409343289 (56h5m45.96081375s) [noise=true , address=145.79.227.33:4881 , piece_id=UM4J3RWOYYEZDPDQAEWPW2LAQ2NZ732HS4VHYJV6GIVNIIS7ZCKA, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34370->145.79.227.33:4881: use of closed network connection Nov 11 16:09:47.409602432 (56h5m45.961072903s) [noise=true , address=145.79.227.27:4707 , piece_id=4GYBLM647IHWXQB63TINHQXJQLKZDGEYWMDSTKFM5ZHD5RMSVSYQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1N7XaDABpimefP5xEz5Swk53fiZk4DTv5L8Hg5cNDgeG6gHP1T, address:145.79.227.27:4707): protocol: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35222->145.79.227.27:4707: use of closed network connection Nov 11 16:09:47.409644141 (56h5m45.961114602s) [noise=true , address=145.79.227.37:4597 , piece_id=2RZEBVXKMEMFLOCYYCBHWYUFULN6AV57ID3YNPODCDRL6QTULKBQ, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:1q6zcreD1FCjZXuYowJz9amfcQtWySL2iyrFLU8BcPzbD8PhPb, address:145.79.227.37:4597): protocol: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34602->145.79.227.37:4597: use of closed network connection Nov 11 16:09:47.409737079 (56h5m45.96120753s) [noise=true , address=145.79.227.44:4228 , piece_id=MF3DSBMBYAPI3A4VNYZOTMCUPQOFG7ALITVFTLVXSI67QZNGGH5Q, seg_pos={0 0}, upload=49413]: Done uploading piece. err: ecclient: upload failed (node:12NBsd33JtAwPdcbD8vBzZUYpjiqZpRzs8y2QrYJFE245pd8T5X, address:145.79.227.44:4228): protocol: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46990->145.79.227.44:4228: use of closed network connection Nov 11 16:09:47.409766335 (56h5m45.961236786s) [seg_pos={0 0}, upload=49413]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Nov 11 16:09:47.658593534 (56h5m46.210063996s) [upload=49413]: Done uploading segments... Nov 11 16:09:47.658685831 (56h5m46.210156282s) [upload=49413]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.923108526s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.............X.] duration: 64.916416ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/79b21962f2d8e182 FAIL (2025-11-11 16:14:59 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:55382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 64.944479ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 587.230293ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/4d1e117d2094896e FAIL (2025-11-11 16:14:33 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:33304->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 587.264358ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........X..........................................XX.....X.] duration: 11.322635428s trace: https://jaeger.dev.storj.io/trace/6ba329aed526607a FAIL (2025-11-11 16:14:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== J6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.204239805 (56h9m48.755710266s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205024630 (56h9m48.756495091s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.205617369 (56h9m48.75708783s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.206216129 (56h9m48.75768659s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.207869137 (56h9m48.759339598s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.209417975 (56h9m48.760888436s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.210891231 (56h9m48.762361693s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.212264837 (56h9m48.763735288s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.213197944 (56h9m48.764668405s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.217083202 (56h9m48.768553663s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.218920661 (56h9m48.770391122s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.220461635 (56h9m48.771932086s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.221453104 (56h9m48.772923565s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.222919085 (56h9m48.774389576s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:50.224462674 (56h9m48.775933135s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Uploading piece... Nov 11 16:13:51.977601348 (56h9m50.52907181s) [upload=49489]: Next returned nil segment Nov 11 16:13:51.977618881 (56h9m50.529089342s) [upload=49489]: Waiting for error group managing segments... Nov 11 16:13:52.318041740 (56h9m50.869512201s) [noise=true , address=77.90.31.19:4843 , piece_id=TZGTZDKKMKAGX2HLFPZJPMI5JPKZ3NIW2MMBIHBDPV4A54T4IWTA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.319228510 (56h9m50.870699001s) [noise=true , address=45.91.116.8:4048 , piece_id=Q2CU5DS5XZAGAVNFIY4ZVGIE446ALQF3MLY53ZP7ZPFPOVAWOTIA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.330974096 (56h9m50.882444547s) [noise=true , address=45.91.116.5:4297 , piece_id=J5RQAJ4REJXDUKZAN6J47FW6H6M63GV4XHEQAQMRJKSW4LUWUXYQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.366779437 (56h9m50.918249898s) [noise=true , address=77.90.31.16:4735 , piece_id=6TZBUPJFXC6LARP6A5V5U4RFSYK6DDNRULNPDBI55A7KCJQO43FQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.403498277 (56h9m50.954968738s) [noise=true , address=45.91.116.13:4155 , piece_id=LHQPEZZOD2ZFPTLEDP7GXEZDSXJNN7NXXGXJEZ3P65MDPDQBE25A, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.410852602 (56h9m50.962323093s) [noise=true , address=45.91.116.7:4009 , piece_id=BU2TOI6TLQGSIDUZRIF447VXGMGWJUQPSS32VMGVGG6JRDZCEEQQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.441652722 (56h9m50.993123183s) [noise=true , address=77.90.31.15:4698 , piece_id=KJZ2GOHKXXWREHUAIMSJ6QQAXZF22QSZDLYNOYZZEJHZOZ7ECQ2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:13:52.943383035 (56h9m51.494853496s) [noise=true , address=45.91.116.9:4261 , piece_id=BYO7FKVEW3T4QEBT3BRHU3PAF6L6Y5WRGTOQHDWWAVQ7MNRMKG2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: <nil> Nov 11 16:14:19.226465189 (56h10m17.77793565s) [noise=true , address=145.79.227.16:4911 , piece_id=UZVJTOTXNOV5ZD3XRBJQJZTCZJXVDNKGPFKS5VIL5QJAWRKSAGYA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54034->145.79.227.16:4911: use of closed network connection Nov 11 16:14:19.226675380 (56h10m17.778145851s) [noise=true , address=145.79.227.25:4617 , piece_id=F6OMLV36XBOM467JQC4M3EQWKNHZMFEDTJEOGKX3XBXYC7USULTQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ifHtxXNY419Q7tJgZbBoz7ZULJmsob3QEgoGTZbhCPXH9TVQw, address:145.79.227.25:4617): protocol: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36394->145.79.227.25:4617: use of closed network connection Nov 11 16:14:19.226719734 (56h10m17.778190195s) [noise=true , address=145.79.227.15:4874 , piece_id=GDFIHSA62E6AZ5UIBDPPHAVCIXK2LSGXVSILV377HS6FUJD2JYEQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36768->145.79.227.15:4874: use of closed network connection Nov 11 16:14:19.226939854 (56h10m17.778410325s) [noise=true , address=145.79.227.27:4689 , piece_id=7KGGGQOLEZXVMSB24VPACO4CT74JIL4X5M5WJMVVMBZWTFMBLGQA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1QXoPNv44qGgVJZR9oRSmogH4paPpS4HX5jLBJdef1X9ev4stQ, address:145.79.227.27:4689): protocol: read tcp 188.166.225.170:46848->145.79.227.27:4689: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.226954371 (56h10m17.778424832s) [noise=true , address=145.79.227.14:4837 , piece_id=BAJBO6ABMXBQSCGJTON4RVNYIWCJW2JMZUE6VQ5KIXXUA5SM7HPA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56454->145.79.227.14:4837: use of closed network connection Nov 11 16:14:19.227194228 (56h10m17.778664699s) [noise=true , address=145.79.227.32:4834 , piece_id=IDANNLR65LHL5JF5ARCKDE7CDAZKTOU7Q6ESIH2JZWG7J36N5WUQ, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1gRnamgAa2ccZS6F44apRmzSkXC6jGvNreJ5J8Y9nciWPZCSsk, address:145.79.227.32:4834): protocol: write tcp 188.166.225.170:47846->145.79.227.32:4834: use of closed network connection; context deadline exceeded Nov 11 16:14:19.227398437 (56h10m17.778868898s) [noise=true , address=145.79.227.11:4731 , piece_id=25Z2HAC7D5QBWTMMZF5FVYR7VPHNU5YPVSXIANUO4OWFSEYHLH2Q, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:46890->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 11 16:14:19.228143486 (56h10m17.779614047s) [noise=true , address=145.79.227.28:4727 , piece_id=ROIVR57UPY33YJRUVW2MH5PPMBLKP3ZBUR5QEW76X6BM5AVWSSDA, seg_pos={0 0}, upload=49489]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35828->145.79.227.28:4727: use of closed network connection Nov 11 16:14:19.228201325 (56h10m17.779671797s) [seg_pos={0 0}, upload=49489]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Nov 11 16:14:19.496444177 (56h10m18.047914638s) [upload=49489]: Done uploading segments... Nov 11 16:14:19.496495415 (56h10m18.047965877s) [upload=49489]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 11.322658021s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)