Orbiter History

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 350.863338ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/fc02574e58740f4 FAIL (2025-11-07 13:22:44 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:52346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 350.893726ms core website
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 4.790741243s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:22:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.790776851s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [........................X..........X........................] duration: 3.464781886s trace: https://jaeger.dev.storj.io/trace/20326ce8f3151a55 FAIL (2025-11-07 13:22:42 UTC) orbiter warning: context deadline exceeded Duration: 3.464839566s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.317621625s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:22:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.317654277s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 329.962157ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/fc02574e58740f4 FAIL (2025-11-07 13:21:39 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:52346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 329.992795ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.........................X..........X.......................] duration: 3.47103304s trace: https://jaeger.dev.storj.io/trace/20326ce8f3151a55 FAIL (2025-11-07 13:21:37 UTC) orbiter warning: context deadline exceeded Duration: 3.471063509s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 4.961729276s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:21:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.961763961s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........................X..........X......................] duration: 3.392340554s trace: https://jaeger.dev.storj.io/trace/20326ce8f3151a55 FAIL (2025-11-07 13:20:46 UTC) orbiter warning: context deadline exceeded Duration: 3.392368377s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.352899317s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:20:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.353063017s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 367.199145ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/fc02574e58740f4 FAIL (2025-11-07 13:20:32 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:52346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 367.228491ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...........................X..........X.....................] duration: 3.417482882s trace: https://jaeger.dev.storj.io/trace/20326ce8f3151a55 FAIL (2025-11-07 13:19:33 UTC) orbiter warning: context deadline exceeded Duration: 3.417515885s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.452586116s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:19:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.452618457s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 347.35141ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/fc02574e58740f4 FAIL (2025-11-07 13:19:25 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:52346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 347.468133ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [............................X..........X....................] duration: 3.417039784s trace: https://jaeger.dev.storj.io/trace/20326ce8f3151a55 FAIL (2025-11-07 13:19:09 UTC) orbiter warning: context deadline exceeded Duration: 3.417072045s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 355.668438ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/fc02574e58740f4 FAIL (2025-11-07 13:18:55 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:52346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 355.696821ms core website
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.840257172s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:18:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.840291706s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.............................X..........X...................] duration: 3.408222325s trace: https://jaeger.dev.storj.io/trace/20326ce8f3151a55 FAIL (2025-11-07 13:18:08 UTC) orbiter warning: context deadline exceeded Duration: 3.408251962s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.810694516s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:18:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.810725726s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 390.022937ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/fc02574e58740f4 FAIL (2025-11-07 13:17:51 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:52346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 390.159126ms core website
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 4.821976131s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:17:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.822007109s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..............................X..........X..................] duration: 3.414957693s trace: https://jaeger.dev.storj.io/trace/20326ce8f3151a55 FAIL (2025-11-07 13:17:05 UTC) orbiter warning: context deadline exceeded Duration: 3.414990154s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 334.632832ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/fc02574e58740f4 FAIL (2025-11-07 13:16:38 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:52346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 334.673258ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...............................X..........X.................] duration: 3.460810456s trace: https://jaeger.dev.storj.io/trace/20326ce8f3151a55 FAIL (2025-11-07 13:16:35 UTC) orbiter warning: context deadline exceeded Duration: 3.460841775s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.413908394s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:16:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.413940095s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 638.061958ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/fc02574e58740f4 FAIL (2025-11-07 13:16:22 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:52346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 638.090633ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.............X.] duration: 367.744223ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/fc02574e58740f4 FAIL (2025-11-07 13:15:45 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:52346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 367.782685ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 68.898316ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/ad5f9081cf270b2 FAIL (2025-11-07 13:15:41 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:43574->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 68.925598ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [................................X..........X................] duration: 3.461288214s trace: https://jaeger.dev.storj.io/trace/20326ce8f3151a55 FAIL (2025-11-07 13:15:23 UTC) orbiter warning: context deadline exceeded Duration: 3.461320624s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.263042753s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:15:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.263074162s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 79.7237ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/ad5f9081cf270b2 FAIL (2025-11-07 13:15:03 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:43574->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 79.755482ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..............X] duration: 644.142152ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/fc02574e58740f4 FAIL (2025-11-07 13:14:43 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:52346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 644.163742ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [X.X............] duration: 74.875648ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/ad5f9081cf270b2 FAIL (2025-11-07 13:14:30 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:43574->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 74.908442ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.................................X..........X...............] duration: 3.513388193s trace: https://jaeger.dev.storj.io/trace/20326ce8f3151a55 FAIL (2025-11-07 13:14:18 UTC) orbiter warning: context deadline exceeded Duration: 3.51342781s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.321592581s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:14:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.32162857s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [.X.X...........] duration: 106.849493ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/ad5f9081cf270b2 FAIL (2025-11-07 13:14:02 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:43574->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 106.87868ms core website
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.173824488s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:13:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.173853363s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..................................X..........X..............] duration: 3.449075186s trace: https://jaeger.dev.storj.io/trace/20326ce8f3151a55 FAIL (2025-11-07 13:13:39 UTC) orbiter warning: context deadline exceeded Duration: 3.449109781s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.803100129s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:13:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.803131018s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.381263738s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:13:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.381299607s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [..X.X..........] duration: 72.289255ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/ad5f9081cf270b2 FAIL (2025-11-07 13:12:52 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:43574->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 72.30802ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...................................X..........X.............] duration: 3.454408285s trace: https://jaeger.dev.storj.io/trace/20326ce8f3151a55 FAIL (2025-11-07 13:12:35 UTC) orbiter warning: context deadline exceeded Duration: 3.454435476s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.417925889s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:12:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.417945466s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [...X.X.........] duration: 101.530485ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/ad5f9081cf270b2 FAIL (2025-11-07 13:12:18 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:43574->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 101.557657ms core website
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 4.768647989s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:11:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.768683066s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [....................................X..........X............] duration: 3.40012125s trace: https://jaeger.dev.storj.io/trace/20326ce8f3151a55 FAIL (2025-11-07 13:11:30 UTC) orbiter warning: context deadline exceeded Duration: 3.400159463s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [....X.X........] duration: 82.894128ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/ad5f9081cf270b2 FAIL (2025-11-07 13:11:22 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:43574->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 82.922371ms core website
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 4.874289786s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:11:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 4.874341133s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [.....X.X.......] duration: 96.954952ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/ad5f9081cf270b2 FAIL (2025-11-07 13:10:51 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:43574->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 96.986223ms core website
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.899512326s trace: https://jaeger.dev.storj.io/trace/525286543fa3ba91 FAIL (2025-11-07 13:10:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 26 but got 25) ======LOGS====== :38.178305861 (192h57m9.874154614s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.178736631 (192h57m9.874585384s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179009922 (192h57m9.874858675s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179196807 (192h57m9.87504557s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179532517 (192h57m9.87538126s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.179619983 (192h57m9.875468726s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180075770 (192h57m9.875924514s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.180093074 (192h57m9.875941827s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.181714923 (192h57m9.877563676s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182276191 (192h57m9.878124944s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.182768168 (192h57m9.878616921s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:38.183246669 (192h57m9.879095422s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Uploading piece... Nov 7 12:37:39.023314018 (192h57m10.719162771s) [upload=170064]: Next returned nil segment Nov 7 12:37:39.023331120 (192h57m10.719179863s) [upload=170064]: Waiting for error group managing segments... Nov 7 12:37:39.404128431 (192h57m11.099977174s) [noise=true , address=81.171.14.70:4032 , piece_id=7NB7BM62JGOHT2ARYQDJZUF67JD7MI6MNEOHR3HMXF6JNFX5KMHQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.409942001 (192h57m11.105790754s) [noise=true , address=81.171.14.70:4045 , piece_id=NCMDOVSYSB46SJYIZ4KS6KN3PQOBBBNKSA53SFWNLLSIFOCJODSQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.418704757 (192h57m11.11455351s) [noise=true , address=81.171.14.70:4047 , piece_id=YHF43Z277LUAXFU57LPF2AG7E5CJYA3YS7EYQ2L3OKUYGV5CA6HA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.422401115 (192h57m11.118249858s) [noise=true , address=81.171.14.70:4033 , piece_id=L6FOGNSLT3QCQD2S2XFBMYTCKBK54A27SFWXEHFNY4KEOZJ663GQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.430155803 (192h57m11.126004546s) [noise=true , address=81.171.14.70:4043 , piece_id=KMPNXQWRIAWQNSYLFRUSDXYYFDWODCNHCZT3M3F3PAGSPVLYIQWA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.432969751 (192h57m11.128818504s) [noise=true , address=81.171.14.70:4035 , piece_id=WKEY3O2VEILUSQOPWLVWUVWTJ5P2FRHFYB3ISJ4CHBRU2PJ4SICQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.456047847 (192h57m11.15189659s) [noise=true , address=89.149.201.147:4006 , piece_id=LASG5H4RJOMUSRW5JLP2VAFTR2Q5Q6WTN5VX5ZH7ZMP35FJFO3LQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.472003520 (192h57m11.167852263s) [noise=true , address=89.149.201.147:4022 , piece_id=O6H2LWVFT7Z6C32QBYT6I3U2BSO5FPPNKJBIKI4TIQZDPTGYLBIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.476240688 (192h57m11.172089442s) [noise=true , address=81.171.14.70:4025 , piece_id=DMUPNB3KVCEVXDDAC42EXY64FAMXEDHCJ7UQ7GXNPMUCTLECIW2A, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.478983470 (192h57m11.174832224s) [noise=true , address=89.149.201.147:4014 , piece_id=A2TBT5YYOEHVDACEULYYDL5E45BBQNFWW4DNQ5KL6XST2H7PKIUA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.490828646 (192h57m11.186677399s) [noise=true , address=89.149.201.147:4004 , piece_id=QJJJNPFD7OK6RNQ4CDVZTULIFNQRUUIMWFN6J6KLYGWPAT2ZX6KQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.499318544 (192h57m11.195167297s) [noise=true , address=81.171.14.70:4044 , piece_id=APHYF67EBVHOUPTQVVRW63TS7DH6GTEQ26KGDECFXXFK5JT2TAGQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.504596154 (192h57m11.200444907s) [noise=true , address=81.171.14.70:4041 , piece_id=B7GD6R6PI7XLRAEMQKLR7NX4URDWMSZSO73NP5PR3MUFLNIWJ3FA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.506012901 (192h57m11.201861654s) [noise=true , address=89.149.201.147:4017 , piece_id=2UZMLY6A7EEOSY4J56C42Q7ASS5KL2RBXQMQGJ7CQ75WXX5VGDAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.519940763 (192h57m11.215789516s) [noise=true , address=89.149.201.147:4012 , piece_id=J7DGFVN7CHOJLDXPOVIOO6H7CCA2NL4GK27TJ5PDQE55N2GXVXFA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:39.539468408 (192h57m11.235317161s) [noise=true , address=89.149.201.147:4019 , piece_id=XTUSCMFNBLMA7H6Y4POVFDW2ELUBFID4QDQI3WYB5NMWAKZWFKUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.677137395 (192h57m12.372986138s) [noise=true , address=185.167.204.206:4049 , piece_id=PCPF6CVHCQ5TQGOTVCG5DYYSIPHPLOM6XSGBFMYSK2PWT4ZKZAPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.716386644 (192h57m12.412235397s) [noise=true , address=185.167.204.206:4043 , piece_id=4AUKA5I7C3CX5U2DRQNT22B7WVJMVSUUWND2HH3TWJPVOZAXSZIQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.724698241 (192h57m12.420546994s) [noise=true , address=185.167.204.205:4020 , piece_id=GGNBFXSOV6GPN5RWDZPZROGFVG5EVMKRWDV5R4CVFCH7M2TBKTNA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.726222003 (192h57m12.422070756s) [noise=true , address=185.167.204.205:4021 , piece_id=LN36HFHPK7AY4B7OYXXSY7JSIW44HLTWR6DOY6OLXR4UME3JHSUQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.727072272 (192h57m12.422921025s) [noise=true , address=185.167.204.205:4032 , piece_id=U7WR7H27PEEVZNN6LATWIKLS5CP5VMJQPW7MVSCRUNRUNFXZRUPQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.735998038 (192h57m12.431846791s) [noise=true , address=185.167.204.205:4023 , piece_id=WSCRJDWKTI2JMLXUVJ7IMTLVXMFGRNCMPWE3U6AVC2J7YS6DEWQQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.788867884 (192h57m12.484716637s) [noise=true , address=185.167.204.206:4052 , piece_id=6TSU5KB4YAQURVO4YEB2OTDHLKJXRBILUVGJSGU4TLRQT7USOPDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.815570602 (192h57m12.511419355s) [noise=true , address=185.167.204.206:4069 , piece_id=CZXHOYZZVBMOUUNDGZBJXHSTTKCGCHEQAQLDOU62JHU3MF4C2JDQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:37:40.833876541 (192h57m12.529725294s) [noise=true , address=185.167.204.205:4033 , piece_id=76TW5MSJ7BNVHNO4OU6ZRAU7OS4NW6QDXQJQ65AVRH45IZ6Q5K3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: <nil> Nov 7 12:38:37.518121189 (192h58m9.213969952s) [noise=true , address=185.167.204.206:4058 , piece_id=QK66LQRH4EM2H5F43KWQITUDXKCZPUS56OUU27BAX755X3LEAV3Q, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:12cJjosS5PjPTn1t4TF4Xw7sL6c2kofhJyiLtitrnDy9V2PZFM7, address:185.167.204.206:4058): protocol: write tcp 188.166.225.170:34970->185.167.204.206:4058: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518260674 (192h58m9.214109427s) [noise=true , address=185.167.204.205:4034 , piece_id=YYHZDPOEDZIOK6ZCEEPZ24YOFLW6FYCUN62SVYBPIQ5RXPV4WRAA, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: write tcp 188.166.225.170:50202->185.167.204.205:4034: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518314648 (192h58m9.214163411s) [noise=true , address=185.167.204.206:4059 , piece_id=CRY5PQKDBMVW2WZ7MW2STHP5GBXWATIC4WSQ6H43Z3JIMXFBEIWQ, seg_pos={0 0}, upload=170064]: Done uploading piece. err: ecclient: upload failed (node:121gTedy8TPx44wmLqdQpZ1gDDsL8D9K1EvQJffR7oKY3bT9DXh, address:185.167.204.206:4059): protocol: rpc: dial tcp 185.167.204.206:4059: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Nov 7 12:38:37.518497015 (192h58m9.214345768s) [seg_pos={0 0}, upload=170064]: Done waiting for segment. successful: 25 optimal: 26 errs: <nil> Nov 7 12:38:37.690636996 (192h58m9.386485749s) [upload=170064]: Done uploading segments... Nov 7 12:38:37.690688734 (192h58m9.386537487s) [upload=170064]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 25) Duration: 5.899542424s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.....................................X..........X...........] duration: 3.423995432s trace: https://jaeger.dev.storj.io/trace/20326ce8f3151a55 FAIL (2025-11-07 13:10:18 UTC) orbiter warning: context deadline exceeded Duration: 3.424023716s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)