Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.................X.....................X...X.X.......X..X...] duration: 5.848729687s trace: https://jaeger.dev.storj.io/trace/62f2c4d5c55fb749 FAIL (2025-12-09 10:47:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.848757529s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 13.431243735s trace: https://jaeger.dev.storj.io/trace/781235978226387b FAIL (2025-12-09 10:47:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ... Dec 9 10:09:19.659155420 (90h40m19.530238455s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.659573456 (90h40m19.530656491s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660082366 (90h40m19.531165401s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660519568 (90h40m19.531602613s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660971178 (90h40m19.532054223s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661377291 (90h40m19.532460326s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661793374 (90h40m19.532876409s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661991682 (90h40m19.533074717s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662431088 (90h40m19.533514133s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662848232 (90h40m19.533931277s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.663338626 (90h40m19.534421671s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.664502282 (90h40m19.535585327s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.666711839 (90h40m19.537794884s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.669766776 (90h40m19.540849821s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.677090573 (90h40m19.548173618s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:21.186498404 (90h40m21.057581499s) [upload=79096]: Next returned nil segment Dec 9 10:09:21.186518313 (90h40m21.057601348s) [upload=79096]: Waiting for error group managing segments... Dec 9 10:09:21.574704328 (90h40m21.445787374s) [noise=true , address=77.90.31.18:4829 , piece_id=CDRLGWU6VF6TU6E4E5B6R7NOF35WOQOA2TSDPASZP424U23QORNA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.587662915 (90h40m21.45874595s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.624695792 (90h40m21.495778827s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.659658729 (90h40m21.530741774s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.667474853 (90h40m21.538557898s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.669766096 (90h40m21.540849131s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.670505664 (90h40m21.541588709s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.680637627 (90h40m21.551720662s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.685855022 (90h40m21.556938057s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.741439516 (90h40m21.612522561s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.306045351 (90h40m22.177128396s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.498367742 (90h40m22.369450787s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:10:18.740403422 (90h41m18.611486467s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1J3HZY416Dm9F5zRXtoDdt8s8tGoU6GpkwS14CXJUzd3zpcYa1, address:45.91.116.13:4173): protocol: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection Dec 9 10:10:18.740531145 (90h41m18.61161419s) [noise=true , address=64.49.13.169:4710 , piece_id=XEKNBKQ4WTECDQD6YYP5RUU2KOSFPRGKXRWDN5THYLE4HXO65BEA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf, address:64.49.13.169:4710): protocol: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection Dec 9 10:10:18.740657375 (90h41m18.61174042s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12wAKmaLHwjBKN8UozGFjGvNMYG57fCxTH8wuDZDRnMdyGBRsTv, address:45.91.116.12:4254): protocol: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection Dec 9 10:10:18.740778986 (90h41m18.611862021s) [noise=true , address=45.91.116.5:4324 , piece_id=XUK3P6OWTGEJMVLR5B3ZR667WEMCEYIAIVW3PTE6ZWTREKSNBTBA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12btBm53zb1MCTy6b7CUeujUruMR9nxBoRgqhHmG5ffPf7rL4F, address:45.91.116.5:4324): protocol: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection Dec 9 10:10:18.740879748 (90h41m18.611962793s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:14DNa2ysTGtfy3Zz72Dq7S3NSGXqnTHZRkuXorDvuuBw1abwMT, address:45.91.116.11:4140): protocol: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection Dec 9 10:10:18.740962677 (90h41m18.612045712s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m, address:45.91.116.9:4264): protocol: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection Dec 9 10:10:18.740989428 (90h41m18.612072463s) [seg_pos={0 0}, upload=79096]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Dec 9 10:10:18.978016295 (90h41m18.84909934s) [upload=79096]: Done uploading segments... Dec 9 10:10:18.978064968 (90h41m18.849148013s) [upload=79096]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 13.431274122s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [......X..X..............X............................X...X..] duration: 3.545286677s trace: https://jaeger.dev.storj.io/trace/507bf9dd5af1d1a FAIL (2025-12-09 10:46:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.545379675s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 317.332745ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Set-Cookie=_cfuvid=7h1mr0Q6MAaMMuyDo3njjnRHVrizchbQDpFC26nhrHc-1765276763664-0.0.1.1-604800000; path=/; domain=.www.storj.io; HttpOnly; Secure; SameSite=None Alt-Svc=h3=":443"; ma=86400 Date=Tue, 09 Dec 2025 10:39:23 GMT Content-Type=text/plain; charset=UTF-8 Server=cloudflare Cf-Ray=9ab3dfe91aee3440-SIN Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT X-Frame-Options=SAMEORIGIN Content-Length=15 Referrer-Policy=same-origin Vary=Accept-Encoding target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/62e446a7fd989959 FAIL (2025-12-09 10:46:39 UTC) orbiter warning: unexpected status code "522 " "error code: 522" Duration: 317.362783ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.936439714s trace: https://jaeger.dev.storj.io/trace/47335bbbc9460f72 FAIL (2025-12-09 10:46:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 9 10:27:57.242720755 (90h58m57.1138038s) [upload=79359]: Starting upload Dec 9 10:27:57.242779226 (90h58m57.113862261s) [upload=79359]: Done starting upload Dec 9 10:27:57.242849329 (90h58m57.113932365s) [upload=79359]: Uploading segments... Dec 9 10:27:57.242961895 (90h58m57.11404493s) [upload=79359]: Got next segment. Inline: false Dec 9 10:27:57.242992913 (90h58m57.114075948s) [upload=79359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 10:27:57.242997352 (90h58m57.114080397s) [upload=79359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 9 10:27:57.490478536 (90h58m57.361561581s) [seg_pos={0 0}, upload=79359]: Begin upload segment... Dec 9 10:27:57.490533320 (90h58m57.361616365s) [seg_pos={0 0}, upload=79359]: Done begin upload segment. Dec 9 10:27:57.490560913 (90h58m57.361643948s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.490634854 (90h58m57.361717889s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491225929 (90h58m57.362308994s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491642873 (90h58m57.362725918s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492140761 (90h58m57.363223796s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492602540 (90h58m57.363685585s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493057557 (90h58m57.364140603s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493071273 (90h58m57.364154318s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493431789 (90h58m57.364514854s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493540356 (90h58m57.364623401s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493892386 (90h58m57.364975431s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494083991 (90h58m57.365167027s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494255959 (90h58m57.365339004s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494606567 (90h58m57.365689602s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494965920 (90h58m57.366048956s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.495345513 (90h58m57.366428558s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:59.193560364 (90h58m59.064643409s) [upload=79359]: Next returned nil segment Dec 9 10:27:59.193597184 (90h58m59.064680219s) [upload=79359]: Waiting for error group managing segments... Dec 9 10:27:59.477920084 (90h58m59.349003119s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.480823943 (90h58m59.351906988s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.487502670 (90h58m59.358585715s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491620402 (90h58m59.362703437s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491964066 (90h58m59.363047101s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.500675514 (90h58m59.371758559s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.501834542 (90h58m59.372917577s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.530610947 (90h58m59.401693992s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.554079847 (90h58m59.425162892s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.558634540 (90h58m59.429717585s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.571829987 (90h58m59.442913023s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.574149323 (90h58m59.445232368s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:28:26.550754779 (90h59m26.421837824s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection Dec 9 10:28:26.550891860 (90h59m26.421974905s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8, address:145.79.227.38:4977): protocol: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection Dec 9 10:28:26.551013401 (90h59m26.422096446s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n, address:145.79.227.40:4051): protocol: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection Dec 9 10:28:26.551092152 (90h59m26.422175187s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection Dec 9 10:28:26.551125365 (90h59m26.42220841s) [seg_pos={0 0}, upload=79359]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 9 10:28:26.791381862 (90h59m26.662464897s) [upload=79359]: Done uploading segments... Dec 9 10:28:26.791420656 (90h59m26.662503691s) [upload=79359]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.936456185s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [.......X..X..............X............................X...X.] duration: 3.495754159s trace: https://jaeger.dev.storj.io/trace/507bf9dd5af1d1a FAIL (2025-12-09 10:46:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.495812368s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 11.056472515s trace: https://jaeger.dev.storj.io/trace/781235978226387b FAIL (2025-12-09 10:46:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ... Dec 9 10:09:19.659155420 (90h40m19.530238455s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.659573456 (90h40m19.530656491s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660082366 (90h40m19.531165401s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660519568 (90h40m19.531602613s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660971178 (90h40m19.532054223s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661377291 (90h40m19.532460326s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661793374 (90h40m19.532876409s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661991682 (90h40m19.533074717s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662431088 (90h40m19.533514133s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662848232 (90h40m19.533931277s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.663338626 (90h40m19.534421671s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.664502282 (90h40m19.535585327s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.666711839 (90h40m19.537794884s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.669766776 (90h40m19.540849821s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.677090573 (90h40m19.548173618s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:21.186498404 (90h40m21.057581499s) [upload=79096]: Next returned nil segment Dec 9 10:09:21.186518313 (90h40m21.057601348s) [upload=79096]: Waiting for error group managing segments... Dec 9 10:09:21.574704328 (90h40m21.445787374s) [noise=true , address=77.90.31.18:4829 , piece_id=CDRLGWU6VF6TU6E4E5B6R7NOF35WOQOA2TSDPASZP424U23QORNA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.587662915 (90h40m21.45874595s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.624695792 (90h40m21.495778827s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.659658729 (90h40m21.530741774s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.667474853 (90h40m21.538557898s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.669766096 (90h40m21.540849131s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.670505664 (90h40m21.541588709s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.680637627 (90h40m21.551720662s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.685855022 (90h40m21.556938057s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.741439516 (90h40m21.612522561s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.306045351 (90h40m22.177128396s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.498367742 (90h40m22.369450787s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:10:18.740403422 (90h41m18.611486467s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1J3HZY416Dm9F5zRXtoDdt8s8tGoU6GpkwS14CXJUzd3zpcYa1, address:45.91.116.13:4173): protocol: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection Dec 9 10:10:18.740531145 (90h41m18.61161419s) [noise=true , address=64.49.13.169:4710 , piece_id=XEKNBKQ4WTECDQD6YYP5RUU2KOSFPRGKXRWDN5THYLE4HXO65BEA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf, address:64.49.13.169:4710): protocol: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection Dec 9 10:10:18.740657375 (90h41m18.61174042s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12wAKmaLHwjBKN8UozGFjGvNMYG57fCxTH8wuDZDRnMdyGBRsTv, address:45.91.116.12:4254): protocol: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection Dec 9 10:10:18.740778986 (90h41m18.611862021s) [noise=true , address=45.91.116.5:4324 , piece_id=XUK3P6OWTGEJMVLR5B3ZR667WEMCEYIAIVW3PTE6ZWTREKSNBTBA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12btBm53zb1MCTy6b7CUeujUruMR9nxBoRgqhHmG5ffPf7rL4F, address:45.91.116.5:4324): protocol: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection Dec 9 10:10:18.740879748 (90h41m18.611962793s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:14DNa2ysTGtfy3Zz72Dq7S3NSGXqnTHZRkuXorDvuuBw1abwMT, address:45.91.116.11:4140): protocol: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection Dec 9 10:10:18.740962677 (90h41m18.612045712s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m, address:45.91.116.9:4264): protocol: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection Dec 9 10:10:18.740989428 (90h41m18.612072463s) [seg_pos={0 0}, upload=79096]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Dec 9 10:10:18.978016295 (90h41m18.84909934s) [upload=79096]: Done uploading segments... Dec 9 10:10:18.978064968 (90h41m18.849148013s) [upload=79096]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 11.056493455s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 319.087758ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Set-Cookie=_cfuvid=7h1mr0Q6MAaMMuyDo3njjnRHVrizchbQDpFC26nhrHc-1765276763664-0.0.1.1-604800000; path=/; domain=.www.storj.io; HttpOnly; Secure; SameSite=None Alt-Svc=h3=":443"; ma=86400 Date=Tue, 09 Dec 2025 10:39:23 GMT Content-Type=text/plain; charset=UTF-8 Server=cloudflare Cf-Ray=9ab3dfe91aee3440-SIN Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT X-Frame-Options=SAMEORIGIN Content-Length=15 Referrer-Policy=same-origin Vary=Accept-Encoding target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/62e446a7fd989959 FAIL (2025-12-09 10:46:16 UTC) orbiter warning: unexpected status code "522 " "error code: 522" Duration: 319.2518ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [..................X.....................X...X.X.......X..X..] duration: 5.75235929s trace: https://jaeger.dev.storj.io/trace/62f2c4d5c55fb749 FAIL (2025-12-09 10:45:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.75238546s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 17.986992561s trace: https://jaeger.dev.storj.io/trace/47335bbbc9460f72 FAIL (2025-12-09 10:45:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 9 10:27:57.242720755 (90h58m57.1138038s) [upload=79359]: Starting upload Dec 9 10:27:57.242779226 (90h58m57.113862261s) [upload=79359]: Done starting upload Dec 9 10:27:57.242849329 (90h58m57.113932365s) [upload=79359]: Uploading segments... Dec 9 10:27:57.242961895 (90h58m57.11404493s) [upload=79359]: Got next segment. Inline: false Dec 9 10:27:57.242992913 (90h58m57.114075948s) [upload=79359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 10:27:57.242997352 (90h58m57.114080397s) [upload=79359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 9 10:27:57.490478536 (90h58m57.361561581s) [seg_pos={0 0}, upload=79359]: Begin upload segment... Dec 9 10:27:57.490533320 (90h58m57.361616365s) [seg_pos={0 0}, upload=79359]: Done begin upload segment. Dec 9 10:27:57.490560913 (90h58m57.361643948s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.490634854 (90h58m57.361717889s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491225929 (90h58m57.362308994s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491642873 (90h58m57.362725918s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492140761 (90h58m57.363223796s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492602540 (90h58m57.363685585s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493057557 (90h58m57.364140603s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493071273 (90h58m57.364154318s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493431789 (90h58m57.364514854s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493540356 (90h58m57.364623401s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493892386 (90h58m57.364975431s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494083991 (90h58m57.365167027s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494255959 (90h58m57.365339004s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494606567 (90h58m57.365689602s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494965920 (90h58m57.366048956s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.495345513 (90h58m57.366428558s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:59.193560364 (90h58m59.064643409s) [upload=79359]: Next returned nil segment Dec 9 10:27:59.193597184 (90h58m59.064680219s) [upload=79359]: Waiting for error group managing segments... Dec 9 10:27:59.477920084 (90h58m59.349003119s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.480823943 (90h58m59.351906988s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.487502670 (90h58m59.358585715s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491620402 (90h58m59.362703437s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491964066 (90h58m59.363047101s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.500675514 (90h58m59.371758559s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.501834542 (90h58m59.372917577s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.530610947 (90h58m59.401693992s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.554079847 (90h58m59.425162892s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.558634540 (90h58m59.429717585s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.571829987 (90h58m59.442913023s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.574149323 (90h58m59.445232368s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:28:26.550754779 (90h59m26.421837824s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection Dec 9 10:28:26.550891860 (90h59m26.421974905s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8, address:145.79.227.38:4977): protocol: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection Dec 9 10:28:26.551013401 (90h59m26.422096446s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n, address:145.79.227.40:4051): protocol: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection Dec 9 10:28:26.551092152 (90h59m26.422175187s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection Dec 9 10:28:26.551125365 (90h59m26.42220841s) [seg_pos={0 0}, upload=79359]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 9 10:28:26.791381862 (90h59m26.662464897s) [upload=79359]: Done uploading segments... Dec 9 10:28:26.791420656 (90h59m26.662503691s) [upload=79359]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 17.987016946s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [........X..X..............X............................X...X] duration: 5.001426941s trace: https://jaeger.dev.storj.io/trace/507bf9dd5af1d1a FAIL (2025-12-09 10:45:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001450888s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 320.789728ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Set-Cookie=_cfuvid=7h1mr0Q6MAaMMuyDo3njjnRHVrizchbQDpFC26nhrHc-1765276763664-0.0.1.1-604800000; path=/; domain=.www.storj.io; HttpOnly; Secure; SameSite=None Alt-Svc=h3=":443"; ma=86400 Date=Tue, 09 Dec 2025 10:39:23 GMT Content-Type=text/plain; charset=UTF-8 Server=cloudflare Cf-Ray=9ab3dfe91aee3440-SIN Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT X-Frame-Options=SAMEORIGIN Content-Length=15 Referrer-Policy=same-origin Vary=Accept-Encoding target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/62e446a7fd989959 FAIL (2025-12-09 10:45:34 UTC) orbiter warning: unexpected status code "522 " "error code: 522" Duration: 320.818544ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [X..................X.....................X...X.X.......X..X.] duration: 5.611064447s trace: https://jaeger.dev.storj.io/trace/62f2c4d5c55fb749 FAIL (2025-12-09 10:45:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.61108151s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 11.663639345s trace: https://jaeger.dev.storj.io/trace/781235978226387b FAIL (2025-12-09 10:45:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ... Dec 9 10:09:19.659155420 (90h40m19.530238455s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.659573456 (90h40m19.530656491s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660082366 (90h40m19.531165401s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660519568 (90h40m19.531602613s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660971178 (90h40m19.532054223s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661377291 (90h40m19.532460326s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661793374 (90h40m19.532876409s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661991682 (90h40m19.533074717s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662431088 (90h40m19.533514133s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662848232 (90h40m19.533931277s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.663338626 (90h40m19.534421671s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.664502282 (90h40m19.535585327s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.666711839 (90h40m19.537794884s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.669766776 (90h40m19.540849821s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.677090573 (90h40m19.548173618s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:21.186498404 (90h40m21.057581499s) [upload=79096]: Next returned nil segment Dec 9 10:09:21.186518313 (90h40m21.057601348s) [upload=79096]: Waiting for error group managing segments... Dec 9 10:09:21.574704328 (90h40m21.445787374s) [noise=true , address=77.90.31.18:4829 , piece_id=CDRLGWU6VF6TU6E4E5B6R7NOF35WOQOA2TSDPASZP424U23QORNA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.587662915 (90h40m21.45874595s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.624695792 (90h40m21.495778827s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.659658729 (90h40m21.530741774s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.667474853 (90h40m21.538557898s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.669766096 (90h40m21.540849131s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.670505664 (90h40m21.541588709s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.680637627 (90h40m21.551720662s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.685855022 (90h40m21.556938057s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.741439516 (90h40m21.612522561s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.306045351 (90h40m22.177128396s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.498367742 (90h40m22.369450787s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:10:18.740403422 (90h41m18.611486467s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1J3HZY416Dm9F5zRXtoDdt8s8tGoU6GpkwS14CXJUzd3zpcYa1, address:45.91.116.13:4173): protocol: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection Dec 9 10:10:18.740531145 (90h41m18.61161419s) [noise=true , address=64.49.13.169:4710 , piece_id=XEKNBKQ4WTECDQD6YYP5RUU2KOSFPRGKXRWDN5THYLE4HXO65BEA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf, address:64.49.13.169:4710): protocol: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection Dec 9 10:10:18.740657375 (90h41m18.61174042s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12wAKmaLHwjBKN8UozGFjGvNMYG57fCxTH8wuDZDRnMdyGBRsTv, address:45.91.116.12:4254): protocol: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection Dec 9 10:10:18.740778986 (90h41m18.611862021s) [noise=true , address=45.91.116.5:4324 , piece_id=XUK3P6OWTGEJMVLR5B3ZR667WEMCEYIAIVW3PTE6ZWTREKSNBTBA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12btBm53zb1MCTy6b7CUeujUruMR9nxBoRgqhHmG5ffPf7rL4F, address:45.91.116.5:4324): protocol: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection Dec 9 10:10:18.740879748 (90h41m18.611962793s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:14DNa2ysTGtfy3Zz72Dq7S3NSGXqnTHZRkuXorDvuuBw1abwMT, address:45.91.116.11:4140): protocol: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection Dec 9 10:10:18.740962677 (90h41m18.612045712s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m, address:45.91.116.9:4264): protocol: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection Dec 9 10:10:18.740989428 (90h41m18.612072463s) [seg_pos={0 0}, upload=79096]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Dec 9 10:10:18.978016295 (90h41m18.84909934s) [upload=79096]: Done uploading segments... Dec 9 10:10:18.978064968 (90h41m18.849148013s) [upload=79096]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 11.663676195s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [.X..................X.....................X...X.X.......X..X] duration: 1m0.002789315s trace: https://jaeger.dev.storj.io/trace/62f2c4d5c55fb749 FAIL (2025-12-09 10:45:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.002815935s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 309.235728ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Set-Cookie=_cfuvid=7h1mr0Q6MAaMMuyDo3njjnRHVrizchbQDpFC26nhrHc-1765276763664-0.0.1.1-604800000; path=/; domain=.www.storj.io; HttpOnly; Secure; SameSite=None Alt-Svc=h3=":443"; ma=86400 Date=Tue, 09 Dec 2025 10:39:23 GMT Content-Type=text/plain; charset=UTF-8 Server=cloudflare Cf-Ray=9ab3dfe91aee3440-SIN Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT X-Frame-Options=SAMEORIGIN Content-Length=15 Referrer-Policy=same-origin Vary=Accept-Encoding target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/62e446a7fd989959 FAIL (2025-12-09 10:45:10 UTC) orbiter warning: unexpected status code "522 " "error code: 522" Duration: 309.287808ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 7.036859175s trace: https://jaeger.dev.storj.io/trace/47335bbbc9460f72 FAIL (2025-12-09 10:44:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 9 10:27:57.242720755 (90h58m57.1138038s) [upload=79359]: Starting upload Dec 9 10:27:57.242779226 (90h58m57.113862261s) [upload=79359]: Done starting upload Dec 9 10:27:57.242849329 (90h58m57.113932365s) [upload=79359]: Uploading segments... Dec 9 10:27:57.242961895 (90h58m57.11404493s) [upload=79359]: Got next segment. Inline: false Dec 9 10:27:57.242992913 (90h58m57.114075948s) [upload=79359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 10:27:57.242997352 (90h58m57.114080397s) [upload=79359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 9 10:27:57.490478536 (90h58m57.361561581s) [seg_pos={0 0}, upload=79359]: Begin upload segment... Dec 9 10:27:57.490533320 (90h58m57.361616365s) [seg_pos={0 0}, upload=79359]: Done begin upload segment. Dec 9 10:27:57.490560913 (90h58m57.361643948s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.490634854 (90h58m57.361717889s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491225929 (90h58m57.362308994s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491642873 (90h58m57.362725918s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492140761 (90h58m57.363223796s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492602540 (90h58m57.363685585s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493057557 (90h58m57.364140603s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493071273 (90h58m57.364154318s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493431789 (90h58m57.364514854s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493540356 (90h58m57.364623401s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493892386 (90h58m57.364975431s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494083991 (90h58m57.365167027s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494255959 (90h58m57.365339004s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494606567 (90h58m57.365689602s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494965920 (90h58m57.366048956s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.495345513 (90h58m57.366428558s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:59.193560364 (90h58m59.064643409s) [upload=79359]: Next returned nil segment Dec 9 10:27:59.193597184 (90h58m59.064680219s) [upload=79359]: Waiting for error group managing segments... Dec 9 10:27:59.477920084 (90h58m59.349003119s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.480823943 (90h58m59.351906988s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.487502670 (90h58m59.358585715s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491620402 (90h58m59.362703437s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491964066 (90h58m59.363047101s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.500675514 (90h58m59.371758559s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.501834542 (90h58m59.372917577s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.530610947 (90h58m59.401693992s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.554079847 (90h58m59.425162892s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.558634540 (90h58m59.429717585s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.571829987 (90h58m59.442913023s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.574149323 (90h58m59.445232368s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:28:26.550754779 (90h59m26.421837824s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection Dec 9 10:28:26.550891860 (90h59m26.421974905s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8, address:145.79.227.38:4977): protocol: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection Dec 9 10:28:26.551013401 (90h59m26.422096446s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n, address:145.79.227.40:4051): protocol: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection Dec 9 10:28:26.551092152 (90h59m26.422175187s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection Dec 9 10:28:26.551125365 (90h59m26.42220841s) [seg_pos={0 0}, upload=79359]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 9 10:28:26.791381862 (90h59m26.662464897s) [upload=79359]: Done uploading segments... Dec 9 10:28:26.791420656 (90h59m26.662503691s) [upload=79359]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.036881027s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.130052582s trace: https://jaeger.dev.storj.io/trace/47335bbbc9460f72 FAIL (2025-12-09 10:44:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 9 10:27:57.242720755 (90h58m57.1138038s) [upload=79359]: Starting upload Dec 9 10:27:57.242779226 (90h58m57.113862261s) [upload=79359]: Done starting upload Dec 9 10:27:57.242849329 (90h58m57.113932365s) [upload=79359]: Uploading segments... Dec 9 10:27:57.242961895 (90h58m57.11404493s) [upload=79359]: Got next segment. Inline: false Dec 9 10:27:57.242992913 (90h58m57.114075948s) [upload=79359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 10:27:57.242997352 (90h58m57.114080397s) [upload=79359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 9 10:27:57.490478536 (90h58m57.361561581s) [seg_pos={0 0}, upload=79359]: Begin upload segment... Dec 9 10:27:57.490533320 (90h58m57.361616365s) [seg_pos={0 0}, upload=79359]: Done begin upload segment. Dec 9 10:27:57.490560913 (90h58m57.361643948s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.490634854 (90h58m57.361717889s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491225929 (90h58m57.362308994s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491642873 (90h58m57.362725918s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492140761 (90h58m57.363223796s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492602540 (90h58m57.363685585s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493057557 (90h58m57.364140603s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493071273 (90h58m57.364154318s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493431789 (90h58m57.364514854s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493540356 (90h58m57.364623401s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493892386 (90h58m57.364975431s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494083991 (90h58m57.365167027s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494255959 (90h58m57.365339004s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494606567 (90h58m57.365689602s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494965920 (90h58m57.366048956s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.495345513 (90h58m57.366428558s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:59.193560364 (90h58m59.064643409s) [upload=79359]: Next returned nil segment Dec 9 10:27:59.193597184 (90h58m59.064680219s) [upload=79359]: Waiting for error group managing segments... Dec 9 10:27:59.477920084 (90h58m59.349003119s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.480823943 (90h58m59.351906988s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.487502670 (90h58m59.358585715s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491620402 (90h58m59.362703437s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491964066 (90h58m59.363047101s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.500675514 (90h58m59.371758559s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.501834542 (90h58m59.372917577s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.530610947 (90h58m59.401693992s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.554079847 (90h58m59.425162892s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.558634540 (90h58m59.429717585s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.571829987 (90h58m59.442913023s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.574149323 (90h58m59.445232368s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:28:26.550754779 (90h59m26.421837824s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection Dec 9 10:28:26.550891860 (90h59m26.421974905s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8, address:145.79.227.38:4977): protocol: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection Dec 9 10:28:26.551013401 (90h59m26.422096446s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n, address:145.79.227.40:4051): protocol: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection Dec 9 10:28:26.551092152 (90h59m26.422175187s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection Dec 9 10:28:26.551125365 (90h59m26.42220841s) [seg_pos={0 0}, upload=79359]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 9 10:28:26.791381862 (90h59m26.662464897s) [upload=79359]: Done uploading segments... Dec 9 10:28:26.791420656 (90h59m26.662503691s) [upload=79359]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.130129959s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [.........X..X..............X............................X...] duration: 3.466160225s trace: https://jaeger.dev.storj.io/trace/3ce1c03cc148cccd FAIL (2025-12-09 10:44:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.466188088s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 10.942853163s trace: https://jaeger.dev.storj.io/trace/781235978226387b FAIL (2025-12-09 10:44:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ... Dec 9 10:09:19.659155420 (90h40m19.530238455s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.659573456 (90h40m19.530656491s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660082366 (90h40m19.531165401s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660519568 (90h40m19.531602613s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660971178 (90h40m19.532054223s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661377291 (90h40m19.532460326s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661793374 (90h40m19.532876409s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661991682 (90h40m19.533074717s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662431088 (90h40m19.533514133s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662848232 (90h40m19.533931277s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.663338626 (90h40m19.534421671s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.664502282 (90h40m19.535585327s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.666711839 (90h40m19.537794884s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.669766776 (90h40m19.540849821s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.677090573 (90h40m19.548173618s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:21.186498404 (90h40m21.057581499s) [upload=79096]: Next returned nil segment Dec 9 10:09:21.186518313 (90h40m21.057601348s) [upload=79096]: Waiting for error group managing segments... Dec 9 10:09:21.574704328 (90h40m21.445787374s) [noise=true , address=77.90.31.18:4829 , piece_id=CDRLGWU6VF6TU6E4E5B6R7NOF35WOQOA2TSDPASZP424U23QORNA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.587662915 (90h40m21.45874595s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.624695792 (90h40m21.495778827s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.659658729 (90h40m21.530741774s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.667474853 (90h40m21.538557898s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.669766096 (90h40m21.540849131s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.670505664 (90h40m21.541588709s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.680637627 (90h40m21.551720662s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.685855022 (90h40m21.556938057s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.741439516 (90h40m21.612522561s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.306045351 (90h40m22.177128396s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.498367742 (90h40m22.369450787s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:10:18.740403422 (90h41m18.611486467s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1J3HZY416Dm9F5zRXtoDdt8s8tGoU6GpkwS14CXJUzd3zpcYa1, address:45.91.116.13:4173): protocol: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection Dec 9 10:10:18.740531145 (90h41m18.61161419s) [noise=true , address=64.49.13.169:4710 , piece_id=XEKNBKQ4WTECDQD6YYP5RUU2KOSFPRGKXRWDN5THYLE4HXO65BEA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf, address:64.49.13.169:4710): protocol: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection Dec 9 10:10:18.740657375 (90h41m18.61174042s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12wAKmaLHwjBKN8UozGFjGvNMYG57fCxTH8wuDZDRnMdyGBRsTv, address:45.91.116.12:4254): protocol: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection Dec 9 10:10:18.740778986 (90h41m18.611862021s) [noise=true , address=45.91.116.5:4324 , piece_id=XUK3P6OWTGEJMVLR5B3ZR667WEMCEYIAIVW3PTE6ZWTREKSNBTBA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12btBm53zb1MCTy6b7CUeujUruMR9nxBoRgqhHmG5ffPf7rL4F, address:45.91.116.5:4324): protocol: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection Dec 9 10:10:18.740879748 (90h41m18.611962793s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:14DNa2ysTGtfy3Zz72Dq7S3NSGXqnTHZRkuXorDvuuBw1abwMT, address:45.91.116.11:4140): protocol: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection Dec 9 10:10:18.740962677 (90h41m18.612045712s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m, address:45.91.116.9:4264): protocol: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection Dec 9 10:10:18.740989428 (90h41m18.612072463s) [seg_pos={0 0}, upload=79096]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Dec 9 10:10:18.978016295 (90h41m18.84909934s) [upload=79096]: Done uploading segments... Dec 9 10:10:18.978064968 (90h41m18.849148013s) [upload=79096]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 10.942871457s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 307.558734ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Set-Cookie=_cfuvid=7h1mr0Q6MAaMMuyDo3njjnRHVrizchbQDpFC26nhrHc-1765276763664-0.0.1.1-604800000; path=/; domain=.www.storj.io; HttpOnly; Secure; SameSite=None Alt-Svc=h3=":443"; ma=86400 Date=Tue, 09 Dec 2025 10:39:23 GMT Content-Type=text/plain; charset=UTF-8 Server=cloudflare Cf-Ray=9ab3dfe91aee3440-SIN Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT X-Frame-Options=SAMEORIGIN Content-Length=15 Referrer-Policy=same-origin Vary=Accept-Encoding target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/62e446a7fd989959 FAIL (2025-12-09 10:44:16 UTC) orbiter warning: unexpected status code "522 " "error code: 522" Duration: 307.589253ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [..X..................X.....................X...X.X.......X..] duration: 5.943326105s trace: https://jaeger.dev.storj.io/trace/efeccdc252f2db8 FAIL (2025-12-09 10:44:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.943355602s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [..........X..X..............X............................X..] duration: 3.440620001s trace: https://jaeger.dev.storj.io/trace/3ce1c03cc148cccd FAIL (2025-12-09 10:44:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.440701146s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 12.262973297s trace: https://jaeger.dev.storj.io/trace/47335bbbc9460f72 FAIL (2025-12-09 10:43:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 9 10:27:57.242720755 (90h58m57.1138038s) [upload=79359]: Starting upload Dec 9 10:27:57.242779226 (90h58m57.113862261s) [upload=79359]: Done starting upload Dec 9 10:27:57.242849329 (90h58m57.113932365s) [upload=79359]: Uploading segments... Dec 9 10:27:57.242961895 (90h58m57.11404493s) [upload=79359]: Got next segment. Inline: false Dec 9 10:27:57.242992913 (90h58m57.114075948s) [upload=79359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 10:27:57.242997352 (90h58m57.114080397s) [upload=79359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 9 10:27:57.490478536 (90h58m57.361561581s) [seg_pos={0 0}, upload=79359]: Begin upload segment... Dec 9 10:27:57.490533320 (90h58m57.361616365s) [seg_pos={0 0}, upload=79359]: Done begin upload segment. Dec 9 10:27:57.490560913 (90h58m57.361643948s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.490634854 (90h58m57.361717889s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491225929 (90h58m57.362308994s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491642873 (90h58m57.362725918s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492140761 (90h58m57.363223796s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492602540 (90h58m57.363685585s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493057557 (90h58m57.364140603s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493071273 (90h58m57.364154318s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493431789 (90h58m57.364514854s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493540356 (90h58m57.364623401s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493892386 (90h58m57.364975431s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494083991 (90h58m57.365167027s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494255959 (90h58m57.365339004s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494606567 (90h58m57.365689602s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494965920 (90h58m57.366048956s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.495345513 (90h58m57.366428558s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:59.193560364 (90h58m59.064643409s) [upload=79359]: Next returned nil segment Dec 9 10:27:59.193597184 (90h58m59.064680219s) [upload=79359]: Waiting for error group managing segments... Dec 9 10:27:59.477920084 (90h58m59.349003119s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.480823943 (90h58m59.351906988s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.487502670 (90h58m59.358585715s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491620402 (90h58m59.362703437s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491964066 (90h58m59.363047101s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.500675514 (90h58m59.371758559s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.501834542 (90h58m59.372917577s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.530610947 (90h58m59.401693992s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.554079847 (90h58m59.425162892s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.558634540 (90h58m59.429717585s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.571829987 (90h58m59.442913023s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.574149323 (90h58m59.445232368s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:28:26.550754779 (90h59m26.421837824s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection Dec 9 10:28:26.550891860 (90h59m26.421974905s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8, address:145.79.227.38:4977): protocol: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection Dec 9 10:28:26.551013401 (90h59m26.422096446s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n, address:145.79.227.40:4051): protocol: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection Dec 9 10:28:26.551092152 (90h59m26.422175187s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection Dec 9 10:28:26.551125365 (90h59m26.42220841s) [seg_pos={0 0}, upload=79359]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 9 10:28:26.791381862 (90h59m26.662464897s) [upload=79359]: Done uploading segments... Dec 9 10:28:26.791420656 (90h59m26.662503691s) [upload=79359]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 12.262994337s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 314.069772ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Set-Cookie=_cfuvid=7h1mr0Q6MAaMMuyDo3njjnRHVrizchbQDpFC26nhrHc-1765276763664-0.0.1.1-604800000; path=/; domain=.www.storj.io; HttpOnly; Secure; SameSite=None Alt-Svc=h3=":443"; ma=86400 Date=Tue, 09 Dec 2025 10:39:23 GMT Content-Type=text/plain; charset=UTF-8 Server=cloudflare Cf-Ray=9ab3dfe91aee3440-SIN Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT X-Frame-Options=SAMEORIGIN Content-Length=15 Referrer-Policy=same-origin Vary=Accept-Encoding target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/62e446a7fd989959 FAIL (2025-12-09 10:43:30 UTC) orbiter warning: unexpected status code "522 " "error code: 522" Duration: 314.091033ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 12.998185291s trace: https://jaeger.dev.storj.io/trace/781235978226387b FAIL (2025-12-09 10:43:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ... Dec 9 10:09:19.659155420 (90h40m19.530238455s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.659573456 (90h40m19.530656491s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660082366 (90h40m19.531165401s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660519568 (90h40m19.531602613s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660971178 (90h40m19.532054223s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661377291 (90h40m19.532460326s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661793374 (90h40m19.532876409s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661991682 (90h40m19.533074717s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662431088 (90h40m19.533514133s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662848232 (90h40m19.533931277s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.663338626 (90h40m19.534421671s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.664502282 (90h40m19.535585327s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.666711839 (90h40m19.537794884s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.669766776 (90h40m19.540849821s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.677090573 (90h40m19.548173618s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:21.186498404 (90h40m21.057581499s) [upload=79096]: Next returned nil segment Dec 9 10:09:21.186518313 (90h40m21.057601348s) [upload=79096]: Waiting for error group managing segments... Dec 9 10:09:21.574704328 (90h40m21.445787374s) [noise=true , address=77.90.31.18:4829 , piece_id=CDRLGWU6VF6TU6E4E5B6R7NOF35WOQOA2TSDPASZP424U23QORNA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.587662915 (90h40m21.45874595s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.624695792 (90h40m21.495778827s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.659658729 (90h40m21.530741774s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.667474853 (90h40m21.538557898s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.669766096 (90h40m21.540849131s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.670505664 (90h40m21.541588709s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.680637627 (90h40m21.551720662s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.685855022 (90h40m21.556938057s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.741439516 (90h40m21.612522561s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.306045351 (90h40m22.177128396s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.498367742 (90h40m22.369450787s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:10:18.740403422 (90h41m18.611486467s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1J3HZY416Dm9F5zRXtoDdt8s8tGoU6GpkwS14CXJUzd3zpcYa1, address:45.91.116.13:4173): protocol: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection Dec 9 10:10:18.740531145 (90h41m18.61161419s) [noise=true , address=64.49.13.169:4710 , piece_id=XEKNBKQ4WTECDQD6YYP5RUU2KOSFPRGKXRWDN5THYLE4HXO65BEA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf, address:64.49.13.169:4710): protocol: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection Dec 9 10:10:18.740657375 (90h41m18.61174042s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12wAKmaLHwjBKN8UozGFjGvNMYG57fCxTH8wuDZDRnMdyGBRsTv, address:45.91.116.12:4254): protocol: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection Dec 9 10:10:18.740778986 (90h41m18.611862021s) [noise=true , address=45.91.116.5:4324 , piece_id=XUK3P6OWTGEJMVLR5B3ZR667WEMCEYIAIVW3PTE6ZWTREKSNBTBA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12btBm53zb1MCTy6b7CUeujUruMR9nxBoRgqhHmG5ffPf7rL4F, address:45.91.116.5:4324): protocol: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection Dec 9 10:10:18.740879748 (90h41m18.611962793s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:14DNa2ysTGtfy3Zz72Dq7S3NSGXqnTHZRkuXorDvuuBw1abwMT, address:45.91.116.11:4140): protocol: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection Dec 9 10:10:18.740962677 (90h41m18.612045712s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m, address:45.91.116.9:4264): protocol: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection Dec 9 10:10:18.740989428 (90h41m18.612072463s) [seg_pos={0 0}, upload=79096]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Dec 9 10:10:18.978016295 (90h41m18.84909934s) [upload=79096]: Done uploading segments... Dec 9 10:10:18.978064968 (90h41m18.849148013s) [upload=79096]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 12.998210549s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [...X..................X.....................X...X.X.......X.] duration: 5.96251251s trace: https://jaeger.dev.storj.io/trace/efeccdc252f2db8 FAIL (2025-12-09 10:43:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.962531467s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [...........X..X..............X............................X.] duration: 3.446834303s trace: https://jaeger.dev.storj.io/trace/3ce1c03cc148cccd FAIL (2025-12-09 10:42:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.446865091s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 15.608027424s trace: https://jaeger.dev.storj.io/trace/781235978226387b FAIL (2025-12-09 10:42:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ... Dec 9 10:09:19.659155420 (90h40m19.530238455s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.659573456 (90h40m19.530656491s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660082366 (90h40m19.531165401s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660519568 (90h40m19.531602613s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660971178 (90h40m19.532054223s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661377291 (90h40m19.532460326s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661793374 (90h40m19.532876409s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661991682 (90h40m19.533074717s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662431088 (90h40m19.533514133s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662848232 (90h40m19.533931277s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.663338626 (90h40m19.534421671s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.664502282 (90h40m19.535585327s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.666711839 (90h40m19.537794884s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.669766776 (90h40m19.540849821s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.677090573 (90h40m19.548173618s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:21.186498404 (90h40m21.057581499s) [upload=79096]: Next returned nil segment Dec 9 10:09:21.186518313 (90h40m21.057601348s) [upload=79096]: Waiting for error group managing segments... Dec 9 10:09:21.574704328 (90h40m21.445787374s) [noise=true , address=77.90.31.18:4829 , piece_id=CDRLGWU6VF6TU6E4E5B6R7NOF35WOQOA2TSDPASZP424U23QORNA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.587662915 (90h40m21.45874595s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.624695792 (90h40m21.495778827s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.659658729 (90h40m21.530741774s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.667474853 (90h40m21.538557898s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.669766096 (90h40m21.540849131s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.670505664 (90h40m21.541588709s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.680637627 (90h40m21.551720662s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.685855022 (90h40m21.556938057s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.741439516 (90h40m21.612522561s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.306045351 (90h40m22.177128396s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.498367742 (90h40m22.369450787s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:10:18.740403422 (90h41m18.611486467s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1J3HZY416Dm9F5zRXtoDdt8s8tGoU6GpkwS14CXJUzd3zpcYa1, address:45.91.116.13:4173): protocol: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection Dec 9 10:10:18.740531145 (90h41m18.61161419s) [noise=true , address=64.49.13.169:4710 , piece_id=XEKNBKQ4WTECDQD6YYP5RUU2KOSFPRGKXRWDN5THYLE4HXO65BEA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf, address:64.49.13.169:4710): protocol: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection Dec 9 10:10:18.740657375 (90h41m18.61174042s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12wAKmaLHwjBKN8UozGFjGvNMYG57fCxTH8wuDZDRnMdyGBRsTv, address:45.91.116.12:4254): protocol: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection Dec 9 10:10:18.740778986 (90h41m18.611862021s) [noise=true , address=45.91.116.5:4324 , piece_id=XUK3P6OWTGEJMVLR5B3ZR667WEMCEYIAIVW3PTE6ZWTREKSNBTBA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12btBm53zb1MCTy6b7CUeujUruMR9nxBoRgqhHmG5ffPf7rL4F, address:45.91.116.5:4324): protocol: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection Dec 9 10:10:18.740879748 (90h41m18.611962793s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:14DNa2ysTGtfy3Zz72Dq7S3NSGXqnTHZRkuXorDvuuBw1abwMT, address:45.91.116.11:4140): protocol: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection Dec 9 10:10:18.740962677 (90h41m18.612045712s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m, address:45.91.116.9:4264): protocol: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection Dec 9 10:10:18.740989428 (90h41m18.612072463s) [seg_pos={0 0}, upload=79096]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Dec 9 10:10:18.978016295 (90h41m18.84909934s) [upload=79096]: Done uploading segments... Dec 9 10:10:18.978064968 (90h41m18.849148013s) [upload=79096]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 15.608058303s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [....X..................X.....................X...X.X.......X] duration: 1m0.001974892s trace: https://jaeger.dev.storj.io/trace/efeccdc252f2db8 FAIL (2025-12-09 10:42:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.00202677s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 15.184532261s trace: https://jaeger.dev.storj.io/trace/781235978226387b FAIL (2025-12-09 10:42:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ... Dec 9 10:09:19.659155420 (90h40m19.530238455s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.659573456 (90h40m19.530656491s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660082366 (90h40m19.531165401s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660519568 (90h40m19.531602613s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660971178 (90h40m19.532054223s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661377291 (90h40m19.532460326s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661793374 (90h40m19.532876409s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661991682 (90h40m19.533074717s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662431088 (90h40m19.533514133s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662848232 (90h40m19.533931277s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.663338626 (90h40m19.534421671s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.664502282 (90h40m19.535585327s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.666711839 (90h40m19.537794884s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.669766776 (90h40m19.540849821s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.677090573 (90h40m19.548173618s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:21.186498404 (90h40m21.057581499s) [upload=79096]: Next returned nil segment Dec 9 10:09:21.186518313 (90h40m21.057601348s) [upload=79096]: Waiting for error group managing segments... Dec 9 10:09:21.574704328 (90h40m21.445787374s) [noise=true , address=77.90.31.18:4829 , piece_id=CDRLGWU6VF6TU6E4E5B6R7NOF35WOQOA2TSDPASZP424U23QORNA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.587662915 (90h40m21.45874595s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.624695792 (90h40m21.495778827s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.659658729 (90h40m21.530741774s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.667474853 (90h40m21.538557898s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.669766096 (90h40m21.540849131s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.670505664 (90h40m21.541588709s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.680637627 (90h40m21.551720662s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.685855022 (90h40m21.556938057s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.741439516 (90h40m21.612522561s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.306045351 (90h40m22.177128396s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.498367742 (90h40m22.369450787s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:10:18.740403422 (90h41m18.611486467s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1J3HZY416Dm9F5zRXtoDdt8s8tGoU6GpkwS14CXJUzd3zpcYa1, address:45.91.116.13:4173): protocol: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection Dec 9 10:10:18.740531145 (90h41m18.61161419s) [noise=true , address=64.49.13.169:4710 , piece_id=XEKNBKQ4WTECDQD6YYP5RUU2KOSFPRGKXRWDN5THYLE4HXO65BEA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf, address:64.49.13.169:4710): protocol: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection Dec 9 10:10:18.740657375 (90h41m18.61174042s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12wAKmaLHwjBKN8UozGFjGvNMYG57fCxTH8wuDZDRnMdyGBRsTv, address:45.91.116.12:4254): protocol: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection Dec 9 10:10:18.740778986 (90h41m18.611862021s) [noise=true , address=45.91.116.5:4324 , piece_id=XUK3P6OWTGEJMVLR5B3ZR667WEMCEYIAIVW3PTE6ZWTREKSNBTBA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12btBm53zb1MCTy6b7CUeujUruMR9nxBoRgqhHmG5ffPf7rL4F, address:45.91.116.5:4324): protocol: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection Dec 9 10:10:18.740879748 (90h41m18.611962793s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:14DNa2ysTGtfy3Zz72Dq7S3NSGXqnTHZRkuXorDvuuBw1abwMT, address:45.91.116.11:4140): protocol: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection Dec 9 10:10:18.740962677 (90h41m18.612045712s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m, address:45.91.116.9:4264): protocol: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection Dec 9 10:10:18.740989428 (90h41m18.612072463s) [seg_pos={0 0}, upload=79096]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Dec 9 10:10:18.978016295 (90h41m18.84909934s) [upload=79096]: Done uploading segments... Dec 9 10:10:18.978064968 (90h41m18.849148013s) [upload=79096]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 15.18455785s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 304.185731ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Set-Cookie=_cfuvid=7h1mr0Q6MAaMMuyDo3njjnRHVrizchbQDpFC26nhrHc-1765276763664-0.0.1.1-604800000; path=/; domain=.www.storj.io; HttpOnly; Secure; SameSite=None Alt-Svc=h3=":443"; ma=86400 Date=Tue, 09 Dec 2025 10:39:23 GMT Content-Type=text/plain; charset=UTF-8 Server=cloudflare Cf-Ray=9ab3dfe91aee3440-SIN Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT X-Frame-Options=SAMEORIGIN Content-Length=15 Referrer-Policy=same-origin Vary=Accept-Encoding target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/62e446a7fd989959 FAIL (2025-12-09 10:42:31 UTC) orbiter warning: unexpected status code "522 " "error code: 522" Duration: 304.221088ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.897686009s trace: https://jaeger.dev.storj.io/trace/47335bbbc9460f72 FAIL (2025-12-09 10:42:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 9 10:27:57.242720755 (90h58m57.1138038s) [upload=79359]: Starting upload Dec 9 10:27:57.242779226 (90h58m57.113862261s) [upload=79359]: Done starting upload Dec 9 10:27:57.242849329 (90h58m57.113932365s) [upload=79359]: Uploading segments... Dec 9 10:27:57.242961895 (90h58m57.11404493s) [upload=79359]: Got next segment. Inline: false Dec 9 10:27:57.242992913 (90h58m57.114075948s) [upload=79359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 10:27:57.242997352 (90h58m57.114080397s) [upload=79359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 9 10:27:57.490478536 (90h58m57.361561581s) [seg_pos={0 0}, upload=79359]: Begin upload segment... Dec 9 10:27:57.490533320 (90h58m57.361616365s) [seg_pos={0 0}, upload=79359]: Done begin upload segment. Dec 9 10:27:57.490560913 (90h58m57.361643948s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.490634854 (90h58m57.361717889s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491225929 (90h58m57.362308994s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491642873 (90h58m57.362725918s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492140761 (90h58m57.363223796s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492602540 (90h58m57.363685585s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493057557 (90h58m57.364140603s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493071273 (90h58m57.364154318s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493431789 (90h58m57.364514854s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493540356 (90h58m57.364623401s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493892386 (90h58m57.364975431s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494083991 (90h58m57.365167027s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494255959 (90h58m57.365339004s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494606567 (90h58m57.365689602s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494965920 (90h58m57.366048956s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.495345513 (90h58m57.366428558s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:59.193560364 (90h58m59.064643409s) [upload=79359]: Next returned nil segment Dec 9 10:27:59.193597184 (90h58m59.064680219s) [upload=79359]: Waiting for error group managing segments... Dec 9 10:27:59.477920084 (90h58m59.349003119s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.480823943 (90h58m59.351906988s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.487502670 (90h58m59.358585715s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491620402 (90h58m59.362703437s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491964066 (90h58m59.363047101s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.500675514 (90h58m59.371758559s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.501834542 (90h58m59.372917577s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.530610947 (90h58m59.401693992s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.554079847 (90h58m59.425162892s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.558634540 (90h58m59.429717585s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.571829987 (90h58m59.442913023s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.574149323 (90h58m59.445232368s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:28:26.550754779 (90h59m26.421837824s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection Dec 9 10:28:26.550891860 (90h59m26.421974905s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8, address:145.79.227.38:4977): protocol: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection Dec 9 10:28:26.551013401 (90h59m26.422096446s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n, address:145.79.227.40:4051): protocol: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection Dec 9 10:28:26.551092152 (90h59m26.422175187s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection Dec 9 10:28:26.551125365 (90h59m26.42220841s) [seg_pos={0 0}, upload=79359]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 9 10:28:26.791381862 (90h59m26.662464897s) [upload=79359]: Done uploading segments... Dec 9 10:28:26.791420656 (90h59m26.662503691s) [upload=79359]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.89771331s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [............X..X..............X............................X] duration: 5.000525394s trace: https://jaeger.dev.storj.io/trace/3ce1c03cc148cccd FAIL (2025-12-09 10:42:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000551233s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 11.863402692s trace: https://jaeger.dev.storj.io/trace/47335bbbc9460f72 FAIL (2025-12-09 10:42:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 9 10:27:57.242720755 (90h58m57.1138038s) [upload=79359]: Starting upload Dec 9 10:27:57.242779226 (90h58m57.113862261s) [upload=79359]: Done starting upload Dec 9 10:27:57.242849329 (90h58m57.113932365s) [upload=79359]: Uploading segments... Dec 9 10:27:57.242961895 (90h58m57.11404493s) [upload=79359]: Got next segment. Inline: false Dec 9 10:27:57.242992913 (90h58m57.114075948s) [upload=79359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 10:27:57.242997352 (90h58m57.114080397s) [upload=79359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 9 10:27:57.490478536 (90h58m57.361561581s) [seg_pos={0 0}, upload=79359]: Begin upload segment... Dec 9 10:27:57.490533320 (90h58m57.361616365s) [seg_pos={0 0}, upload=79359]: Done begin upload segment. Dec 9 10:27:57.490560913 (90h58m57.361643948s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.490634854 (90h58m57.361717889s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491225929 (90h58m57.362308994s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491642873 (90h58m57.362725918s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492140761 (90h58m57.363223796s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492602540 (90h58m57.363685585s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493057557 (90h58m57.364140603s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493071273 (90h58m57.364154318s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493431789 (90h58m57.364514854s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493540356 (90h58m57.364623401s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493892386 (90h58m57.364975431s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494083991 (90h58m57.365167027s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494255959 (90h58m57.365339004s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494606567 (90h58m57.365689602s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494965920 (90h58m57.366048956s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.495345513 (90h58m57.366428558s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:59.193560364 (90h58m59.064643409s) [upload=79359]: Next returned nil segment Dec 9 10:27:59.193597184 (90h58m59.064680219s) [upload=79359]: Waiting for error group managing segments... Dec 9 10:27:59.477920084 (90h58m59.349003119s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.480823943 (90h58m59.351906988s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.487502670 (90h58m59.358585715s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491620402 (90h58m59.362703437s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491964066 (90h58m59.363047101s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.500675514 (90h58m59.371758559s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.501834542 (90h58m59.372917577s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.530610947 (90h58m59.401693992s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.554079847 (90h58m59.425162892s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.558634540 (90h58m59.429717585s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.571829987 (90h58m59.442913023s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.574149323 (90h58m59.445232368s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:28:26.550754779 (90h59m26.421837824s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection Dec 9 10:28:26.550891860 (90h59m26.421974905s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8, address:145.79.227.38:4977): protocol: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection Dec 9 10:28:26.551013401 (90h59m26.422096446s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n, address:145.79.227.40:4051): protocol: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection Dec 9 10:28:26.551092152 (90h59m26.422175187s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection Dec 9 10:28:26.551125365 (90h59m26.42220841s) [seg_pos={0 0}, upload=79359]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 9 10:28:26.791381862 (90h59m26.662464897s) [upload=79359]: Done uploading segments... Dec 9 10:28:26.791420656 (90h59m26.662503691s) [upload=79359]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 11.863456714s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 14.691755774s trace: https://jaeger.dev.storj.io/trace/781235978226387b FAIL (2025-12-09 10:42:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ... Dec 9 10:09:19.659155420 (90h40m19.530238455s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.659573456 (90h40m19.530656491s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660082366 (90h40m19.531165401s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660519568 (90h40m19.531602613s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660971178 (90h40m19.532054223s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661377291 (90h40m19.532460326s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661793374 (90h40m19.532876409s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661991682 (90h40m19.533074717s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662431088 (90h40m19.533514133s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662848232 (90h40m19.533931277s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.663338626 (90h40m19.534421671s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.664502282 (90h40m19.535585327s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.666711839 (90h40m19.537794884s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.669766776 (90h40m19.540849821s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.677090573 (90h40m19.548173618s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:21.186498404 (90h40m21.057581499s) [upload=79096]: Next returned nil segment Dec 9 10:09:21.186518313 (90h40m21.057601348s) [upload=79096]: Waiting for error group managing segments... Dec 9 10:09:21.574704328 (90h40m21.445787374s) [noise=true , address=77.90.31.18:4829 , piece_id=CDRLGWU6VF6TU6E4E5B6R7NOF35WOQOA2TSDPASZP424U23QORNA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.587662915 (90h40m21.45874595s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.624695792 (90h40m21.495778827s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.659658729 (90h40m21.530741774s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.667474853 (90h40m21.538557898s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.669766096 (90h40m21.540849131s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.670505664 (90h40m21.541588709s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.680637627 (90h40m21.551720662s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.685855022 (90h40m21.556938057s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.741439516 (90h40m21.612522561s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.306045351 (90h40m22.177128396s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.498367742 (90h40m22.369450787s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:10:18.740403422 (90h41m18.611486467s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1J3HZY416Dm9F5zRXtoDdt8s8tGoU6GpkwS14CXJUzd3zpcYa1, address:45.91.116.13:4173): protocol: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection Dec 9 10:10:18.740531145 (90h41m18.61161419s) [noise=true , address=64.49.13.169:4710 , piece_id=XEKNBKQ4WTECDQD6YYP5RUU2KOSFPRGKXRWDN5THYLE4HXO65BEA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf, address:64.49.13.169:4710): protocol: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection Dec 9 10:10:18.740657375 (90h41m18.61174042s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12wAKmaLHwjBKN8UozGFjGvNMYG57fCxTH8wuDZDRnMdyGBRsTv, address:45.91.116.12:4254): protocol: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection Dec 9 10:10:18.740778986 (90h41m18.611862021s) [noise=true , address=45.91.116.5:4324 , piece_id=XUK3P6OWTGEJMVLR5B3ZR667WEMCEYIAIVW3PTE6ZWTREKSNBTBA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12btBm53zb1MCTy6b7CUeujUruMR9nxBoRgqhHmG5ffPf7rL4F, address:45.91.116.5:4324): protocol: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection Dec 9 10:10:18.740879748 (90h41m18.611962793s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:14DNa2ysTGtfy3Zz72Dq7S3NSGXqnTHZRkuXorDvuuBw1abwMT, address:45.91.116.11:4140): protocol: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection Dec 9 10:10:18.740962677 (90h41m18.612045712s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m, address:45.91.116.9:4264): protocol: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection Dec 9 10:10:18.740989428 (90h41m18.612072463s) [seg_pos={0 0}, upload=79096]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Dec 9 10:10:18.978016295 (90h41m18.84909934s) [upload=79096]: Done uploading segments... Dec 9 10:10:18.978064968 (90h41m18.849148013s) [upload=79096]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 14.69178487s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.............X..X..............X............................] duration: 4.311092608s trace: https://jaeger.dev.storj.io/trace/9e23c2a25a964f2 FAIL (2025-12-09 10:41:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.311160245s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 307.990575ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Set-Cookie=_cfuvid=7h1mr0Q6MAaMMuyDo3njjnRHVrizchbQDpFC26nhrHc-1765276763664-0.0.1.1-604800000; path=/; domain=.www.storj.io; HttpOnly; Secure; SameSite=None Alt-Svc=h3=":443"; ma=86400 Date=Tue, 09 Dec 2025 10:39:23 GMT Content-Type=text/plain; charset=UTF-8 Server=cloudflare Cf-Ray=9ab3dfe91aee3440-SIN Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT X-Frame-Options=SAMEORIGIN Content-Length=15 Referrer-Policy=same-origin Vary=Accept-Encoding target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/62e446a7fd989959 FAIL (2025-12-09 10:41:47 UTC) orbiter warning: unexpected status code "522 " "error code: 522" Duration: 308.023458ms core website
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [.....X..................X.....................X...X.X.......] duration: 5.790720366s trace: https://jaeger.dev.storj.io/trace/58ad150942c6dac6 FAIL (2025-12-09 10:41:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.790760613s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [..............X..X..............X...........................] duration: 3.494959046s trace: https://jaeger.dev.storj.io/trace/9e23c2a25a964f2 FAIL (2025-12-09 10:41:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.494986799s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 10.784526754s trace: https://jaeger.dev.storj.io/trace/47335bbbc9460f72 FAIL (2025-12-09 10:41:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 9 10:27:57.242720755 (90h58m57.1138038s) [upload=79359]: Starting upload Dec 9 10:27:57.242779226 (90h58m57.113862261s) [upload=79359]: Done starting upload Dec 9 10:27:57.242849329 (90h58m57.113932365s) [upload=79359]: Uploading segments... Dec 9 10:27:57.242961895 (90h58m57.11404493s) [upload=79359]: Got next segment. Inline: false Dec 9 10:27:57.242992913 (90h58m57.114075948s) [upload=79359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 10:27:57.242997352 (90h58m57.114080397s) [upload=79359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 9 10:27:57.490478536 (90h58m57.361561581s) [seg_pos={0 0}, upload=79359]: Begin upload segment... Dec 9 10:27:57.490533320 (90h58m57.361616365s) [seg_pos={0 0}, upload=79359]: Done begin upload segment. Dec 9 10:27:57.490560913 (90h58m57.361643948s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.490634854 (90h58m57.361717889s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491225929 (90h58m57.362308994s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491642873 (90h58m57.362725918s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492140761 (90h58m57.363223796s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492602540 (90h58m57.363685585s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493057557 (90h58m57.364140603s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493071273 (90h58m57.364154318s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493431789 (90h58m57.364514854s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493540356 (90h58m57.364623401s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493892386 (90h58m57.364975431s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494083991 (90h58m57.365167027s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494255959 (90h58m57.365339004s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494606567 (90h58m57.365689602s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494965920 (90h58m57.366048956s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.495345513 (90h58m57.366428558s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:59.193560364 (90h58m59.064643409s) [upload=79359]: Next returned nil segment Dec 9 10:27:59.193597184 (90h58m59.064680219s) [upload=79359]: Waiting for error group managing segments... Dec 9 10:27:59.477920084 (90h58m59.349003119s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.480823943 (90h58m59.351906988s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.487502670 (90h58m59.358585715s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491620402 (90h58m59.362703437s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491964066 (90h58m59.363047101s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.500675514 (90h58m59.371758559s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.501834542 (90h58m59.372917577s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.530610947 (90h58m59.401693992s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.554079847 (90h58m59.425162892s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.558634540 (90h58m59.429717585s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.571829987 (90h58m59.442913023s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.574149323 (90h58m59.445232368s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:28:26.550754779 (90h59m26.421837824s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection Dec 9 10:28:26.550891860 (90h59m26.421974905s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8, address:145.79.227.38:4977): protocol: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection Dec 9 10:28:26.551013401 (90h59m26.422096446s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n, address:145.79.227.40:4051): protocol: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection Dec 9 10:28:26.551092152 (90h59m26.422175187s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection Dec 9 10:28:26.551125365 (90h59m26.42220841s) [seg_pos={0 0}, upload=79359]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 9 10:28:26.791381862 (90h59m26.662464897s) [upload=79359]: Done uploading segments... Dec 9 10:28:26.791420656 (90h59m26.662503691s) [upload=79359]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 10.784552845s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 11.621675506s trace: https://jaeger.dev.storj.io/trace/781235978226387b FAIL (2025-12-09 10:41:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ... Dec 9 10:09:19.659155420 (90h40m19.530238455s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.659573456 (90h40m19.530656491s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660082366 (90h40m19.531165401s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660519568 (90h40m19.531602613s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660971178 (90h40m19.532054223s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661377291 (90h40m19.532460326s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661793374 (90h40m19.532876409s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661991682 (90h40m19.533074717s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662431088 (90h40m19.533514133s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662848232 (90h40m19.533931277s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.663338626 (90h40m19.534421671s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.664502282 (90h40m19.535585327s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.666711839 (90h40m19.537794884s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.669766776 (90h40m19.540849821s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.677090573 (90h40m19.548173618s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:21.186498404 (90h40m21.057581499s) [upload=79096]: Next returned nil segment Dec 9 10:09:21.186518313 (90h40m21.057601348s) [upload=79096]: Waiting for error group managing segments... Dec 9 10:09:21.574704328 (90h40m21.445787374s) [noise=true , address=77.90.31.18:4829 , piece_id=CDRLGWU6VF6TU6E4E5B6R7NOF35WOQOA2TSDPASZP424U23QORNA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.587662915 (90h40m21.45874595s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.624695792 (90h40m21.495778827s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.659658729 (90h40m21.530741774s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.667474853 (90h40m21.538557898s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.669766096 (90h40m21.540849131s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.670505664 (90h40m21.541588709s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.680637627 (90h40m21.551720662s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.685855022 (90h40m21.556938057s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.741439516 (90h40m21.612522561s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.306045351 (90h40m22.177128396s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.498367742 (90h40m22.369450787s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:10:18.740403422 (90h41m18.611486467s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1J3HZY416Dm9F5zRXtoDdt8s8tGoU6GpkwS14CXJUzd3zpcYa1, address:45.91.116.13:4173): protocol: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection Dec 9 10:10:18.740531145 (90h41m18.61161419s) [noise=true , address=64.49.13.169:4710 , piece_id=XEKNBKQ4WTECDQD6YYP5RUU2KOSFPRGKXRWDN5THYLE4HXO65BEA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf, address:64.49.13.169:4710): protocol: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection Dec 9 10:10:18.740657375 (90h41m18.61174042s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12wAKmaLHwjBKN8UozGFjGvNMYG57fCxTH8wuDZDRnMdyGBRsTv, address:45.91.116.12:4254): protocol: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection Dec 9 10:10:18.740778986 (90h41m18.611862021s) [noise=true , address=45.91.116.5:4324 , piece_id=XUK3P6OWTGEJMVLR5B3ZR667WEMCEYIAIVW3PTE6ZWTREKSNBTBA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12btBm53zb1MCTy6b7CUeujUruMR9nxBoRgqhHmG5ffPf7rL4F, address:45.91.116.5:4324): protocol: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection Dec 9 10:10:18.740879748 (90h41m18.611962793s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:14DNa2ysTGtfy3Zz72Dq7S3NSGXqnTHZRkuXorDvuuBw1abwMT, address:45.91.116.11:4140): protocol: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection Dec 9 10:10:18.740962677 (90h41m18.612045712s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m, address:45.91.116.9:4264): protocol: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection Dec 9 10:10:18.740989428 (90h41m18.612072463s) [seg_pos={0 0}, upload=79096]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Dec 9 10:10:18.978016295 (90h41m18.84909934s) [upload=79096]: Done uploading segments... Dec 9 10:10:18.978064968 (90h41m18.849148013s) [upload=79096]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 11.62170978s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 318.745053ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Set-Cookie=_cfuvid=7h1mr0Q6MAaMMuyDo3njjnRHVrizchbQDpFC26nhrHc-1765276763664-0.0.1.1-604800000; path=/; domain=.www.storj.io; HttpOnly; Secure; SameSite=None Alt-Svc=h3=":443"; ma=86400 Date=Tue, 09 Dec 2025 10:39:23 GMT Content-Type=text/plain; charset=UTF-8 Server=cloudflare Cf-Ray=9ab3dfe91aee3440-SIN Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT X-Frame-Options=SAMEORIGIN Content-Length=15 Referrer-Policy=same-origin Vary=Accept-Encoding target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/62e446a7fd989959 FAIL (2025-12-09 10:41:09 UTC) orbiter warning: unexpected status code "522 " "error code: 522" Duration: 318.775232ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.955383904s trace: https://jaeger.dev.storj.io/trace/6cb2657ad006bedf FAIL (2025-12-09 10:41:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 9 09:57:44.389846478 (90h28m44.260929523s) [upload=78917]: Starting upload Dec 9 09:57:44.390051578 (90h28m44.261134613s) [upload=78917]: Done starting upload Dec 9 09:57:44.390125740 (90h28m44.261208775s) [upload=78917]: Uploading segments... Dec 9 09:57:44.390224027 (90h28m44.261307072s) [upload=78917]: Got next segment. Inline: true Dec 9 09:57:44.390230800 (90h28m44.261313835s) [upload=78917]: Waiting for error group managing segments... Dec 9 09:57:44.390312034 (90h28m44.261395069s) [upload=78917]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 09:57:44.390318787 (90h28m44.261401822s) [upload=78917]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Dec 9 09:57:44.390323356 (90h28m44.261406391s) [upload=78917]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 09:58:21.934924620 (90h29m21.806007665s) [upload=78917]: Done uploading segments... Dec 9 09:58:21.934974756 (90h29m21.806057801s) [upload=78917]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.955417326s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 5 out of 60 is failed (threshold=3) [......X..................X.....................X...X.X......] duration: 5.928995078s trace: https://jaeger.dev.storj.io/trace/58ad150942c6dac6 FAIL (2025-12-09 10:41:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.929024404s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 12.076918787s trace: https://jaeger.dev.storj.io/trace/781235978226387b FAIL (2025-12-09 10:40:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ... Dec 9 10:09:19.659155420 (90h40m19.530238455s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.659573456 (90h40m19.530656491s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660082366 (90h40m19.531165401s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660519568 (90h40m19.531602613s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.660971178 (90h40m19.532054223s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661377291 (90h40m19.532460326s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661793374 (90h40m19.532876409s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.661991682 (90h40m19.533074717s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662431088 (90h40m19.533514133s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.662848232 (90h40m19.533931277s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.663338626 (90h40m19.534421671s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.664502282 (90h40m19.535585327s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.666711839 (90h40m19.537794884s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.669766776 (90h40m19.540849821s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:19.677090573 (90h40m19.548173618s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Uploading piece... Dec 9 10:09:21.186498404 (90h40m21.057581499s) [upload=79096]: Next returned nil segment Dec 9 10:09:21.186518313 (90h40m21.057601348s) [upload=79096]: Waiting for error group managing segments... Dec 9 10:09:21.574704328 (90h40m21.445787374s) [noise=true , address=77.90.31.18:4829 , piece_id=CDRLGWU6VF6TU6E4E5B6R7NOF35WOQOA2TSDPASZP424U23QORNA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.587662915 (90h40m21.45874595s) [noise=true , address=77.90.31.10:4529 , piece_id=YIT3CCLUOLSQ3NGOUVF7PYVV6FIDT7WKTRQVU4VTU4LRMPYL4H3A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.624695792 (90h40m21.495778827s) [noise=true , address=77.90.31.15:4703 , piece_id=3FIJGK2XHNRTDF4Q3DN3AX2P42AUR5CNUI5QPXQ256MPMCQB5QRQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.659658729 (90h40m21.530741774s) [noise=true , address=77.90.31.12:4615 , piece_id=BLNHDHE34WZMROFSL5SNTHOITFWI6JOR7VKQBYZB77LTINEGLS5Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.667474853 (90h40m21.538557898s) [noise=true , address=145.79.227.24:4587 , piece_id=EVUNUBMSMMIERRCDR67LDFSGY6S2EF6IRV6EGEX4IOFX4PCTYIDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.669766096 (90h40m21.540849131s) [noise=true , address=77.90.31.14:4684 , piece_id=ARECBM73CDZDYSVWPLH2XCQC6A45SGW57TPWX5URGCAUW355COOA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.670505664 (90h40m21.541588709s) [noise=true , address=77.90.31.17:4783 , piece_id=LSTC3TY566OQBGL6ZXBLL5IFNPORZRWOEGFKGT3EXYOB3IPWI6JQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.680637627 (90h40m21.551720662s) [noise=true , address=77.90.31.13:4639 , piece_id=I2GHDHJTAHCDYP7BAG6QAHGPAFIKQZOLUPD5V7AICS4CATG3XNDA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.685855022 (90h40m21.556938057s) [noise=true , address=77.90.31.21:4882 , piece_id=CJSACAOIT5IXGJXTL4YDNRUK4H5KD6QCFE6VINQGMQJ2TT2TAU5A, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:21.741439516 (90h40m21.612522561s) [noise=true , address=77.90.31.19:4870 , piece_id=P6W2QVXDH57FK755LD67JDSC23BZPHVQHBVWZLJDC457JPG3ZQLQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.306045351 (90h40m22.177128396s) [noise=true , address=77.90.31.16:4760 , piece_id=LJPEX76BQ4ICDJ3I4AWHE7UE7D2WIZCVGQS6XU6ZZ2II2NBYDKHQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:09:22.498367742 (90h40m22.369450787s) [noise=true , address=145.79.227.13:4824 , piece_id=M6P4NFRFOU5LGE2HRGUXRUZXJ2HJYA6BWC3TW4MWUWIYYSUTR76Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: <nil> Dec 9 10:10:18.740403422 (90h41m18.611486467s) [noise=true , address=45.91.116.13:4173 , piece_id=7PIVANLRS7SL6ERBMUC4LT4P3C3JRHRY6Z4I7336EVMXC3IHFVDQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1J3HZY416Dm9F5zRXtoDdt8s8tGoU6GpkwS14CXJUzd3zpcYa1, address:45.91.116.13:4173): protocol: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53476->45.91.116.13:4173: use of closed network connection Dec 9 10:10:18.740531145 (90h41m18.61161419s) [noise=true , address=64.49.13.169:4710 , piece_id=XEKNBKQ4WTECDQD6YYP5RUU2KOSFPRGKXRWDN5THYLE4HXO65BEA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf, address:64.49.13.169:4710): protocol: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37916->64.49.13.169:4710: use of closed network connection Dec 9 10:10:18.740657375 (90h41m18.61174042s) [noise=true , address=45.91.116.12:4254 , piece_id=WOGMNSLO5FOGUSOWXRTLPZNASJB2ZMPUEKC4WRQNROTKQ74JV6KQ, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12wAKmaLHwjBKN8UozGFjGvNMYG57fCxTH8wuDZDRnMdyGBRsTv, address:45.91.116.12:4254): protocol: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35550->45.91.116.12:4254: use of closed network connection Dec 9 10:10:18.740778986 (90h41m18.611862021s) [noise=true , address=45.91.116.5:4324 , piece_id=XUK3P6OWTGEJMVLR5B3ZR667WEMCEYIAIVW3PTE6ZWTREKSNBTBA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:12btBm53zb1MCTy6b7CUeujUruMR9nxBoRgqhHmG5ffPf7rL4F, address:45.91.116.5:4324): protocol: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41676->45.91.116.5:4324: use of closed network connection Dec 9 10:10:18.740879748 (90h41m18.611962793s) [noise=true , address=45.91.116.11:4140 , piece_id=UJVOUUVBOSA4I4YZJIAPP2QGWBUV322ZSSWINU52U5RIVJ4CEZXA, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:14DNa2ysTGtfy3Zz72Dq7S3NSGXqnTHZRkuXorDvuuBw1abwMT, address:45.91.116.11:4140): protocol: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34050->45.91.116.11:4140: use of closed network connection Dec 9 10:10:18.740962677 (90h41m18.612045712s) [noise=true , address=45.91.116.9:4264 , piece_id=7UODMOFUXQZ3KYCY4RFA7OUVYXQTWLKNSWR3QIVMIT5KHEQMGT2Q, seg_pos={0 0}, upload=79096]: Done uploading piece. err: ecclient: upload failed (node:128f6i7ALVXF3Vr8G8zsZXrNBFiT3cK8UAiic5XbFS1gRjZK86m, address:45.91.116.9:4264): protocol: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50170->45.91.116.9:4264: use of closed network connection Dec 9 10:10:18.740989428 (90h41m18.612072463s) [seg_pos={0 0}, upload=79096]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Dec 9 10:10:18.978016295 (90h41m18.84909934s) [upload=79096]: Done uploading segments... Dec 9 10:10:18.978064968 (90h41m18.849148013s) [upload=79096]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 12.076947653s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [...............X..X..............X..........................] duration: 3.519148388s trace: https://jaeger.dev.storj.io/trace/9e23c2a25a964f2 FAIL (2025-12-09 10:40:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.51918081s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 310.426714ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Set-Cookie=_cfuvid=7h1mr0Q6MAaMMuyDo3njjnRHVrizchbQDpFC26nhrHc-1765276763664-0.0.1.1-604800000; path=/; domain=.www.storj.io; HttpOnly; Secure; SameSite=None Alt-Svc=h3=":443"; ma=86400 Date=Tue, 09 Dec 2025 10:39:23 GMT Content-Type=text/plain; charset=UTF-8 Server=cloudflare Cf-Ray=9ab3dfe91aee3440-SIN Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Expires=Thu, 01 Jan 1970 00:00:01 GMT X-Frame-Options=SAMEORIGIN Content-Length=15 Referrer-Policy=same-origin Vary=Accept-Encoding target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/62e446a7fd989959 FAIL (2025-12-09 10:40:36 UTC) orbiter warning: unexpected status code "522 " "error code: 522" Duration: 310.457974ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.967524572s trace: https://jaeger.dev.storj.io/trace/6cb2657ad006bedf FAIL (2025-12-09 10:40:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 9 09:57:44.389846478 (90h28m44.260929523s) [upload=78917]: Starting upload Dec 9 09:57:44.390051578 (90h28m44.261134613s) [upload=78917]: Done starting upload Dec 9 09:57:44.390125740 (90h28m44.261208775s) [upload=78917]: Uploading segments... Dec 9 09:57:44.390224027 (90h28m44.261307072s) [upload=78917]: Got next segment. Inline: true Dec 9 09:57:44.390230800 (90h28m44.261313835s) [upload=78917]: Waiting for error group managing segments... Dec 9 09:57:44.390312034 (90h28m44.261395069s) [upload=78917]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 09:57:44.390318787 (90h28m44.261401822s) [upload=78917]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Dec 9 09:57:44.390323356 (90h28m44.261406391s) [upload=78917]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 09:58:21.934924620 (90h29m21.806007665s) [upload=78917]: Done uploading segments... Dec 9 09:58:21.934974756 (90h29m21.806057801s) [upload=78917]: Upload finished. err: metaclient: context deadline exceeded Duration:* 1.967555891s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.28010828s trace: https://jaeger.dev.storj.io/trace/47335bbbc9460f72 FAIL (2025-12-09 10:40:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 9 10:27:57.242720755 (90h58m57.1138038s) [upload=79359]: Starting upload Dec 9 10:27:57.242779226 (90h58m57.113862261s) [upload=79359]: Done starting upload Dec 9 10:27:57.242849329 (90h58m57.113932365s) [upload=79359]: Uploading segments... Dec 9 10:27:57.242961895 (90h58m57.11404493s) [upload=79359]: Got next segment. Inline: false Dec 9 10:27:57.242992913 (90h58m57.114075948s) [upload=79359]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 10:27:57.242997352 (90h58m57.114080397s) [upload=79359]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 9 10:27:57.490478536 (90h58m57.361561581s) [seg_pos={0 0}, upload=79359]: Begin upload segment... Dec 9 10:27:57.490533320 (90h58m57.361616365s) [seg_pos={0 0}, upload=79359]: Done begin upload segment. Dec 9 10:27:57.490560913 (90h58m57.361643948s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.490634854 (90h58m57.361717889s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491225929 (90h58m57.362308994s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.491642873 (90h58m57.362725918s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492140761 (90h58m57.363223796s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.492602540 (90h58m57.363685585s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493057557 (90h58m57.364140603s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493071273 (90h58m57.364154318s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493431789 (90h58m57.364514854s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493540356 (90h58m57.364623401s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.493892386 (90h58m57.364975431s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494083991 (90h58m57.365167027s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494255959 (90h58m57.365339004s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494606567 (90h58m57.365689602s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.494965920 (90h58m57.366048956s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:57.495345513 (90h58m57.366428558s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Uploading piece... Dec 9 10:27:59.193560364 (90h58m59.064643409s) [upload=79359]: Next returned nil segment Dec 9 10:27:59.193597184 (90h58m59.064680219s) [upload=79359]: Waiting for error group managing segments... Dec 9 10:27:59.477920084 (90h58m59.349003119s) [noise=true , address=145.79.227.42:4121 , piece_id=ERCQT36FQSO22HNFY5QZOJJMERRASCVHJW5G4OKCEJB32UNYUQAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.480823943 (90h58m59.351906988s) [noise=true , address=145.79.227.31:4799 , piece_id=AN4NPL2MIY3NEKKR4TZ7STWCUIZVY6MUQ36D5PJROGD2UG722P2A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.487502670 (90h58m59.358585715s) [noise=true , address=145.79.227.27:4691 , piece_id=7GAKOVJNXFEN5CDKVGK46EIZ6OU4JI247M6Q35WVF4JOWQN72JPA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491620402 (90h58m59.362703437s) [noise=true , address=145.79.227.29:4941 , piece_id=RJUQZUTVF3KJEGZ4MDOP5RR4DEQH5X4OHHXAK3W64CGQOUY4HZWA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.491964066 (90h58m59.363047101s) [noise=true , address=145.79.227.42:4123 , piece_id=6JHXIPA352SVNLEWKKPMANX5MQRHXFURZR753T74MMFTPPK4LHJA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.500675514 (90h58m59.371758559s) [noise=true , address=145.79.227.25:4618 , piece_id=UMDOX46LAA6YKCG74OXMIUAVVFG7MNE3XHL2PBERUOASXEBYMGSQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.501834542 (90h58m59.372917577s) [noise=true , address=145.79.227.39:4014 , piece_id=EW67DYNN54WK4Y7YP7HAZUN227LYDM3LAGCFH7OBET7YIEAUTF3A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.530610947 (90h58m59.401693992s) [noise=true , address=145.79.227.20:4439 , piece_id=JE2X5VYQRHXJDWQFUT2KWCE6UWMD6VLUEHMBZSLL3O6FYMGEY7UQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.554079847 (90h58m59.425162892s) [noise=true , address=145.79.227.22:4511 , piece_id=R5QAVASG7T7UEXDTFXZOK5BPGO7LJARFI7A77MASBGBQJ4DBP4MA, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.558634540 (90h58m59.429717585s) [noise=true , address=145.79.227.24:4583 , piece_id=XSC6J57R4UDC2AL3I7ZK73FLNQG7EIIHIJGR34WAK3LT3U6UZJAQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.571829987 (90h58m59.442913023s) [noise=true , address=145.79.227.32:4835 , piece_id=HVGEKOXP5H2HHK3IG2VTIGXX6UTQKWBX63VKL5VQGQMFIRXQV6BQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:27:59.574149323 (90h58m59.445232368s) [noise=true , address=145.79.227.34:4907 , piece_id=PLUYUJWUQXB6BOP6H6YKCRQ4NNOHJ3M4JHNB7FVKQHO75OJU5HYQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: <nil> Dec 9 10:28:26.550754779 (90h59m26.421837824s) [noise=true , address=45.91.116.8:4047 , piece_id=XOO3P7YG5DW5SYP4OVSAGLKCZ6W3KSVXIUK24GDYOXX2H3HMS5GQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58652->45.91.116.8:4047: use of closed network connection Dec 9 10:28:26.550891860 (90h59m26.421974905s) [noise=true , address=145.79.227.38:4977 , piece_id=5TTHLFLJJXTFEHA2CSV4DCCOMKT5OH7TQPR63TTB57S7O6L4Y73A, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:125dJwj1tvBkFrgDCoj3FhpH553yGgjxPuSTGxbrkGeDwdKk6M8, address:145.79.227.38:4977): protocol: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48912->145.79.227.38:4977: use of closed network connection Dec 9 10:28:26.551013401 (90h59m26.422096446s) [noise=true , address=145.79.227.40:4051 , piece_id=GBINALPVGCPMYGIYXJOXKZFKQK7ERXVAWTEZ6XMPEQMVTRMRYRWQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n, address:145.79.227.40:4051): protocol: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49228->145.79.227.40:4051: use of closed network connection Dec 9 10:28:26.551092152 (90h59m26.422175187s) [noise=true , address=145.79.227.23:4547 , piece_id=SQG4BHO3ZZUORDL6YOPFQNGCKVF7TDJEONIEJWNYGCCFXDWX6KIQ, seg_pos={0 0}, upload=79359]: Done uploading piece. err: ecclient: upload failed (node:1UjJbodRCRm3E63TrzfhbjwFpCmCtfCLS34sQFi1gDJbEwJiKW, address:145.79.227.23:4547): protocol: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60386->145.79.227.23:4547: use of closed network connection Dec 9 10:28:26.551125365 (90h59m26.42220841s) [seg_pos={0 0}, upload=79359]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 9 10:28:26.791381862 (90h59m26.662464897s) [upload=79359]: Done uploading segments... Dec 9 10:28:26.791420656 (90h59m26.662503691s) [upload=79359]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.280140942s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)