Orbiter History

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 315.169515ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/48774610bbd09e70 FAIL (2025-10-31 20:55:55 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 315.2245ms core website
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X....X.....] duration: 6.186115021s trace: https://jaeger.dev.storj.io/trace/35569b98f076d00a FAIL (2025-10-31 20:55:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.186152603s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................................XX....X........] duration: 5.920636894s trace: https://jaeger.dev.storj.io/trace/61feae49d47bc60e FAIL (2025-10-31 20:55:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 20:51:11.706455020 (33h10m43.402303773s) [upload=29117]: Starting upload Oct 31 20:51:11.706524262 (33h10m43.402373005s) [upload=29117]: Done starting upload Oct 31 20:51:11.706833080 (33h10m43.402681823s) [upload=29117]: Uploading segments... Oct 31 20:51:11.707027190 (33h10m43.402875943s) [upload=29117]: Got next segment. Inline: false Oct 31 20:51:11.707074009 (33h10m43.402922762s) [upload=29117]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 20:51:11.707095790 (33h10m43.402944544s) [upload=29117]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 20:51:11.959151951 (33h10m43.655000704s) [seg_pos={0 0}, upload=29117]: Begin upload segment... Oct 31 20:51:11.959219510 (33h10m43.655068263s) [seg_pos={0 0}, upload=29117]: Done begin upload segment. Oct 31 20:51:11.959257482 (33h10m43.655106235s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959352193 (33h10m43.655200936s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959984507 (33h10m43.65583325s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960349932 (33h10m43.656198685s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960434654 (33h10m43.656283397s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960753571 (33h10m43.656602314s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960782867 (33h10m43.65663162s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961093809 (33h10m43.656942562s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961223666 (33h10m43.657072409s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961396354 (33h10m43.657245097s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961606384 (33h10m43.657455127s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961727265 (33h10m43.657576008s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962038738 (33h10m43.657887481s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962095516 (33h10m43.657944259s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962430774 (33h10m43.658279517s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962491260 (33h10m43.658340013s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:13.589873545 (33h10m45.285722288s) [upload=29117]: Next returned nil segment Oct 31 20:51:13.589889816 (33h10m45.285738569s) [upload=29117]: Waiting for error group managing segments... Oct 31 20:51:13.973997709 (33h10m45.669846462s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.977395970 (33h10m45.673244713s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.979447918 (33h10m45.675296671s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.980901436 (33h10m45.676750189s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.986457284 (33h10m45.682306088s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.004274321 (33h10m45.700123084s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.012149448 (33h10m45.707998201s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.027225746 (33h10m45.723074499s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.043308651 (33h10m45.739157404s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.050118539 (33h10m45.745967292s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.065628894 (33h10m45.761477637s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.079898897 (33h10m45.77574765s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:41.002915508 (33h11m12.698764281s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection Oct 31 20:51:41.003131980 (33h11m12.698980723s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection Oct 31 20:51:41.003350967 (33h11m12.69919973s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection Oct 31 20:51:41.003851250 (33h11m12.699700003s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: read tcp 188.166.225.170:49810->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:51:41.003903128 (33h11m12.699751891s) [seg_pos={0 0}, upload=29117]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 20:51:41.248997100 (33h11m12.944845853s) [upload=29117]: Done uploading segments... Oct 31 20:51:41.249037957 (33h11m12.944886711s) [upload=29117]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.920665909s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 313.975005ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/48774610bbd09e70 FAIL (2025-10-31 20:55:12 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 314.007206ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................................XX....X.......] duration: 6.076646058s trace: https://jaeger.dev.storj.io/trace/61feae49d47bc60e FAIL (2025-10-31 20:54:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 20:51:11.706455020 (33h10m43.402303773s) [upload=29117]: Starting upload Oct 31 20:51:11.706524262 (33h10m43.402373005s) [upload=29117]: Done starting upload Oct 31 20:51:11.706833080 (33h10m43.402681823s) [upload=29117]: Uploading segments... Oct 31 20:51:11.707027190 (33h10m43.402875943s) [upload=29117]: Got next segment. Inline: false Oct 31 20:51:11.707074009 (33h10m43.402922762s) [upload=29117]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 20:51:11.707095790 (33h10m43.402944544s) [upload=29117]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 20:51:11.959151951 (33h10m43.655000704s) [seg_pos={0 0}, upload=29117]: Begin upload segment... Oct 31 20:51:11.959219510 (33h10m43.655068263s) [seg_pos={0 0}, upload=29117]: Done begin upload segment. Oct 31 20:51:11.959257482 (33h10m43.655106235s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959352193 (33h10m43.655200936s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959984507 (33h10m43.65583325s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960349932 (33h10m43.656198685s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960434654 (33h10m43.656283397s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960753571 (33h10m43.656602314s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960782867 (33h10m43.65663162s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961093809 (33h10m43.656942562s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961223666 (33h10m43.657072409s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961396354 (33h10m43.657245097s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961606384 (33h10m43.657455127s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961727265 (33h10m43.657576008s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962038738 (33h10m43.657887481s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962095516 (33h10m43.657944259s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962430774 (33h10m43.658279517s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962491260 (33h10m43.658340013s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:13.589873545 (33h10m45.285722288s) [upload=29117]: Next returned nil segment Oct 31 20:51:13.589889816 (33h10m45.285738569s) [upload=29117]: Waiting for error group managing segments... Oct 31 20:51:13.973997709 (33h10m45.669846462s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.977395970 (33h10m45.673244713s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.979447918 (33h10m45.675296671s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.980901436 (33h10m45.676750189s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.986457284 (33h10m45.682306088s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.004274321 (33h10m45.700123084s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.012149448 (33h10m45.707998201s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.027225746 (33h10m45.723074499s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.043308651 (33h10m45.739157404s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.050118539 (33h10m45.745967292s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.065628894 (33h10m45.761477637s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.079898897 (33h10m45.77574765s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:41.002915508 (33h11m12.698764281s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection Oct 31 20:51:41.003131980 (33h11m12.698980723s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection Oct 31 20:51:41.003350967 (33h11m12.69919973s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection Oct 31 20:51:41.003851250 (33h11m12.699700003s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: read tcp 188.166.225.170:49810->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:51:41.003903128 (33h11m12.699751891s) [seg_pos={0 0}, upload=29117]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 20:51:41.248997100 (33h11m12.944845853s) [upload=29117]: Done uploading segments... Oct 31 20:51:41.249037957 (33h11m12.944886711s) [upload=29117]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.076667138s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X....X....] duration: 6.104467612s trace: https://jaeger.dev.storj.io/trace/35569b98f076d00a FAIL (2025-10-31 20:54:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.104497497s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X....X...] duration: 6.461106408s trace: https://jaeger.dev.storj.io/trace/35569b98f076d00a FAIL (2025-10-31 20:54:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.461134751s edge gateway us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 301.265524ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/48774610bbd09e70 FAIL (2025-10-31 20:54:12 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 301.300541ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 336.295771ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/48774610bbd09e70 FAIL (2025-10-31 20:53:40 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 336.315718ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................................XX....X......] duration: 6.107394456s trace: https://jaeger.dev.storj.io/trace/61feae49d47bc60e FAIL (2025-10-31 20:53:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 20:51:11.706455020 (33h10m43.402303773s) [upload=29117]: Starting upload Oct 31 20:51:11.706524262 (33h10m43.402373005s) [upload=29117]: Done starting upload Oct 31 20:51:11.706833080 (33h10m43.402681823s) [upload=29117]: Uploading segments... Oct 31 20:51:11.707027190 (33h10m43.402875943s) [upload=29117]: Got next segment. Inline: false Oct 31 20:51:11.707074009 (33h10m43.402922762s) [upload=29117]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 20:51:11.707095790 (33h10m43.402944544s) [upload=29117]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 20:51:11.959151951 (33h10m43.655000704s) [seg_pos={0 0}, upload=29117]: Begin upload segment... Oct 31 20:51:11.959219510 (33h10m43.655068263s) [seg_pos={0 0}, upload=29117]: Done begin upload segment. Oct 31 20:51:11.959257482 (33h10m43.655106235s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959352193 (33h10m43.655200936s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959984507 (33h10m43.65583325s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960349932 (33h10m43.656198685s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960434654 (33h10m43.656283397s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960753571 (33h10m43.656602314s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960782867 (33h10m43.65663162s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961093809 (33h10m43.656942562s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961223666 (33h10m43.657072409s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961396354 (33h10m43.657245097s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961606384 (33h10m43.657455127s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961727265 (33h10m43.657576008s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962038738 (33h10m43.657887481s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962095516 (33h10m43.657944259s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962430774 (33h10m43.658279517s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962491260 (33h10m43.658340013s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:13.589873545 (33h10m45.285722288s) [upload=29117]: Next returned nil segment Oct 31 20:51:13.589889816 (33h10m45.285738569s) [upload=29117]: Waiting for error group managing segments... Oct 31 20:51:13.973997709 (33h10m45.669846462s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.977395970 (33h10m45.673244713s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.979447918 (33h10m45.675296671s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.980901436 (33h10m45.676750189s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.986457284 (33h10m45.682306088s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.004274321 (33h10m45.700123084s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.012149448 (33h10m45.707998201s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.027225746 (33h10m45.723074499s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.043308651 (33h10m45.739157404s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.050118539 (33h10m45.745967292s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.065628894 (33h10m45.761477637s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.079898897 (33h10m45.77574765s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:41.002915508 (33h11m12.698764281s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection Oct 31 20:51:41.003131980 (33h11m12.698980723s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection Oct 31 20:51:41.003350967 (33h11m12.69919973s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection Oct 31 20:51:41.003851250 (33h11m12.699700003s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: read tcp 188.166.225.170:49810->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:51:41.003903128 (33h11m12.699751891s) [seg_pos={0 0}, upload=29117]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 20:51:41.248997100 (33h11m12.944845853s) [upload=29117]: Done uploading segments... Oct 31 20:51:41.249037957 (33h11m12.944886711s) [upload=29117]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.107429181s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................................XX....X.....] duration: 5.820637634s trace: https://jaeger.dev.storj.io/trace/61feae49d47bc60e FAIL (2025-10-31 20:53:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 20:51:11.706455020 (33h10m43.402303773s) [upload=29117]: Starting upload Oct 31 20:51:11.706524262 (33h10m43.402373005s) [upload=29117]: Done starting upload Oct 31 20:51:11.706833080 (33h10m43.402681823s) [upload=29117]: Uploading segments... Oct 31 20:51:11.707027190 (33h10m43.402875943s) [upload=29117]: Got next segment. Inline: false Oct 31 20:51:11.707074009 (33h10m43.402922762s) [upload=29117]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 20:51:11.707095790 (33h10m43.402944544s) [upload=29117]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 20:51:11.959151951 (33h10m43.655000704s) [seg_pos={0 0}, upload=29117]: Begin upload segment... Oct 31 20:51:11.959219510 (33h10m43.655068263s) [seg_pos={0 0}, upload=29117]: Done begin upload segment. Oct 31 20:51:11.959257482 (33h10m43.655106235s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959352193 (33h10m43.655200936s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959984507 (33h10m43.65583325s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960349932 (33h10m43.656198685s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960434654 (33h10m43.656283397s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960753571 (33h10m43.656602314s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960782867 (33h10m43.65663162s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961093809 (33h10m43.656942562s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961223666 (33h10m43.657072409s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961396354 (33h10m43.657245097s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961606384 (33h10m43.657455127s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961727265 (33h10m43.657576008s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962038738 (33h10m43.657887481s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962095516 (33h10m43.657944259s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962430774 (33h10m43.658279517s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962491260 (33h10m43.658340013s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:13.589873545 (33h10m45.285722288s) [upload=29117]: Next returned nil segment Oct 31 20:51:13.589889816 (33h10m45.285738569s) [upload=29117]: Waiting for error group managing segments... Oct 31 20:51:13.973997709 (33h10m45.669846462s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.977395970 (33h10m45.673244713s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.979447918 (33h10m45.675296671s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.980901436 (33h10m45.676750189s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.986457284 (33h10m45.682306088s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.004274321 (33h10m45.700123084s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.012149448 (33h10m45.707998201s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.027225746 (33h10m45.723074499s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.043308651 (33h10m45.739157404s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.050118539 (33h10m45.745967292s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.065628894 (33h10m45.761477637s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.079898897 (33h10m45.77574765s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:41.002915508 (33h11m12.698764281s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection Oct 31 20:51:41.003131980 (33h11m12.698980723s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection Oct 31 20:51:41.003350967 (33h11m12.69919973s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection Oct 31 20:51:41.003851250 (33h11m12.699700003s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: read tcp 188.166.225.170:49810->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:51:41.003903128 (33h11m12.699751891s) [seg_pos={0 0}, upload=29117]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 20:51:41.248997100 (33h11m12.944845853s) [upload=29117]: Done uploading segments... Oct 31 20:51:41.249037957 (33h11m12.944886711s) [upload=29117]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.820665286s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 304.334138ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/48774610bbd09e70 FAIL (2025-10-31 20:53:11 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 304.36149ms core website
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X....X..] duration: 6.109323174s trace: https://jaeger.dev.storj.io/trace/35569b98f076d00a FAIL (2025-10-31 20:53:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.109354564s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................................XX....X....] duration: 5.818905689s trace: https://jaeger.dev.storj.io/trace/61feae49d47bc60e FAIL (2025-10-31 20:52:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 20:51:11.706455020 (33h10m43.402303773s) [upload=29117]: Starting upload Oct 31 20:51:11.706524262 (33h10m43.402373005s) [upload=29117]: Done starting upload Oct 31 20:51:11.706833080 (33h10m43.402681823s) [upload=29117]: Uploading segments... Oct 31 20:51:11.707027190 (33h10m43.402875943s) [upload=29117]: Got next segment. Inline: false Oct 31 20:51:11.707074009 (33h10m43.402922762s) [upload=29117]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 20:51:11.707095790 (33h10m43.402944544s) [upload=29117]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 20:51:11.959151951 (33h10m43.655000704s) [seg_pos={0 0}, upload=29117]: Begin upload segment... Oct 31 20:51:11.959219510 (33h10m43.655068263s) [seg_pos={0 0}, upload=29117]: Done begin upload segment. Oct 31 20:51:11.959257482 (33h10m43.655106235s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959352193 (33h10m43.655200936s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959984507 (33h10m43.65583325s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960349932 (33h10m43.656198685s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960434654 (33h10m43.656283397s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960753571 (33h10m43.656602314s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960782867 (33h10m43.65663162s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961093809 (33h10m43.656942562s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961223666 (33h10m43.657072409s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961396354 (33h10m43.657245097s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961606384 (33h10m43.657455127s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961727265 (33h10m43.657576008s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962038738 (33h10m43.657887481s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962095516 (33h10m43.657944259s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962430774 (33h10m43.658279517s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962491260 (33h10m43.658340013s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:13.589873545 (33h10m45.285722288s) [upload=29117]: Next returned nil segment Oct 31 20:51:13.589889816 (33h10m45.285738569s) [upload=29117]: Waiting for error group managing segments... Oct 31 20:51:13.973997709 (33h10m45.669846462s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.977395970 (33h10m45.673244713s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.979447918 (33h10m45.675296671s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.980901436 (33h10m45.676750189s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.986457284 (33h10m45.682306088s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.004274321 (33h10m45.700123084s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.012149448 (33h10m45.707998201s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.027225746 (33h10m45.723074499s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.043308651 (33h10m45.739157404s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.050118539 (33h10m45.745967292s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.065628894 (33h10m45.761477637s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.079898897 (33h10m45.77574765s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:41.002915508 (33h11m12.698764281s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection Oct 31 20:51:41.003131980 (33h11m12.698980723s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection Oct 31 20:51:41.003350967 (33h11m12.69919973s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection Oct 31 20:51:41.003851250 (33h11m12.699700003s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: read tcp 188.166.225.170:49810->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:51:41.003903128 (33h11m12.699751891s) [seg_pos={0 0}, upload=29117]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 20:51:41.248997100 (33h11m12.944845853s) [upload=29117]: Done uploading segments... Oct 31 20:51:41.249037957 (33h11m12.944886711s) [upload=29117]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.81893791s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................................XX....X...] duration: 5.893165544s trace: https://jaeger.dev.storj.io/trace/61feae49d47bc60e FAIL (2025-10-31 20:52:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 20:51:11.706455020 (33h10m43.402303773s) [upload=29117]: Starting upload Oct 31 20:51:11.706524262 (33h10m43.402373005s) [upload=29117]: Done starting upload Oct 31 20:51:11.706833080 (33h10m43.402681823s) [upload=29117]: Uploading segments... Oct 31 20:51:11.707027190 (33h10m43.402875943s) [upload=29117]: Got next segment. Inline: false Oct 31 20:51:11.707074009 (33h10m43.402922762s) [upload=29117]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 20:51:11.707095790 (33h10m43.402944544s) [upload=29117]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 20:51:11.959151951 (33h10m43.655000704s) [seg_pos={0 0}, upload=29117]: Begin upload segment... Oct 31 20:51:11.959219510 (33h10m43.655068263s) [seg_pos={0 0}, upload=29117]: Done begin upload segment. Oct 31 20:51:11.959257482 (33h10m43.655106235s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959352193 (33h10m43.655200936s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959984507 (33h10m43.65583325s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960349932 (33h10m43.656198685s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960434654 (33h10m43.656283397s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960753571 (33h10m43.656602314s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960782867 (33h10m43.65663162s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961093809 (33h10m43.656942562s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961223666 (33h10m43.657072409s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961396354 (33h10m43.657245097s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961606384 (33h10m43.657455127s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961727265 (33h10m43.657576008s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962038738 (33h10m43.657887481s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962095516 (33h10m43.657944259s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962430774 (33h10m43.658279517s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962491260 (33h10m43.658340013s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:13.589873545 (33h10m45.285722288s) [upload=29117]: Next returned nil segment Oct 31 20:51:13.589889816 (33h10m45.285738569s) [upload=29117]: Waiting for error group managing segments... Oct 31 20:51:13.973997709 (33h10m45.669846462s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.977395970 (33h10m45.673244713s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.979447918 (33h10m45.675296671s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.980901436 (33h10m45.676750189s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.986457284 (33h10m45.682306088s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.004274321 (33h10m45.700123084s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.012149448 (33h10m45.707998201s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.027225746 (33h10m45.723074499s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.043308651 (33h10m45.739157404s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.050118539 (33h10m45.745967292s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.065628894 (33h10m45.761477637s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.079898897 (33h10m45.77574765s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:41.002915508 (33h11m12.698764281s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection Oct 31 20:51:41.003131980 (33h11m12.698980723s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection Oct 31 20:51:41.003350967 (33h11m12.69919973s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection Oct 31 20:51:41.003851250 (33h11m12.699700003s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: read tcp 188.166.225.170:49810->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:51:41.003903128 (33h11m12.699751891s) [seg_pos={0 0}, upload=29117]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 20:51:41.248997100 (33h11m12.944845853s) [upload=29117]: Done uploading segments... Oct 31 20:51:41.249037957 (33h11m12.944886711s) [upload=29117]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.893193648s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 319.586751ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/48774610bbd09e70 FAIL (2025-10-31 20:52:11 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 319.741706ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................................XX....X..] duration: 6.008548669s trace: https://jaeger.dev.storj.io/trace/61feae49d47bc60e FAIL (2025-10-31 20:52:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 20:51:11.706455020 (33h10m43.402303773s) [upload=29117]: Starting upload Oct 31 20:51:11.706524262 (33h10m43.402373005s) [upload=29117]: Done starting upload Oct 31 20:51:11.706833080 (33h10m43.402681823s) [upload=29117]: Uploading segments... Oct 31 20:51:11.707027190 (33h10m43.402875943s) [upload=29117]: Got next segment. Inline: false Oct 31 20:51:11.707074009 (33h10m43.402922762s) [upload=29117]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 20:51:11.707095790 (33h10m43.402944544s) [upload=29117]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 20:51:11.959151951 (33h10m43.655000704s) [seg_pos={0 0}, upload=29117]: Begin upload segment... Oct 31 20:51:11.959219510 (33h10m43.655068263s) [seg_pos={0 0}, upload=29117]: Done begin upload segment. Oct 31 20:51:11.959257482 (33h10m43.655106235s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959352193 (33h10m43.655200936s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959984507 (33h10m43.65583325s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960349932 (33h10m43.656198685s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960434654 (33h10m43.656283397s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960753571 (33h10m43.656602314s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960782867 (33h10m43.65663162s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961093809 (33h10m43.656942562s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961223666 (33h10m43.657072409s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961396354 (33h10m43.657245097s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961606384 (33h10m43.657455127s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961727265 (33h10m43.657576008s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962038738 (33h10m43.657887481s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962095516 (33h10m43.657944259s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962430774 (33h10m43.658279517s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962491260 (33h10m43.658340013s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:13.589873545 (33h10m45.285722288s) [upload=29117]: Next returned nil segment Oct 31 20:51:13.589889816 (33h10m45.285738569s) [upload=29117]: Waiting for error group managing segments... Oct 31 20:51:13.973997709 (33h10m45.669846462s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.977395970 (33h10m45.673244713s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.979447918 (33h10m45.675296671s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.980901436 (33h10m45.676750189s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.986457284 (33h10m45.682306088s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.004274321 (33h10m45.700123084s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.012149448 (33h10m45.707998201s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.027225746 (33h10m45.723074499s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.043308651 (33h10m45.739157404s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.050118539 (33h10m45.745967292s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.065628894 (33h10m45.761477637s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.079898897 (33h10m45.77574765s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:41.002915508 (33h11m12.698764281s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection Oct 31 20:51:41.003131980 (33h11m12.698980723s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection Oct 31 20:51:41.003350967 (33h11m12.69919973s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection Oct 31 20:51:41.003851250 (33h11m12.699700003s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: read tcp 188.166.225.170:49810->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:51:41.003903128 (33h11m12.699751891s) [seg_pos={0 0}, upload=29117]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 20:51:41.248997100 (33h11m12.944845853s) [upload=29117]: Done uploading segments... Oct 31 20:51:41.249037957 (33h11m12.944886711s) [upload=29117]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.008584738s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X....X.] duration: 6.365300997s trace: https://jaeger.dev.storj.io/trace/35569b98f076d00a FAIL (2025-10-31 20:51:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.365335272s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................................XX....X.] duration: 6.437788156s trace: https://jaeger.dev.storj.io/trace/61feae49d47bc60e FAIL (2025-10-31 20:51:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 20:51:11.706455020 (33h10m43.402303773s) [upload=29117]: Starting upload Oct 31 20:51:11.706524262 (33h10m43.402373005s) [upload=29117]: Done starting upload Oct 31 20:51:11.706833080 (33h10m43.402681823s) [upload=29117]: Uploading segments... Oct 31 20:51:11.707027190 (33h10m43.402875943s) [upload=29117]: Got next segment. Inline: false Oct 31 20:51:11.707074009 (33h10m43.402922762s) [upload=29117]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 20:51:11.707095790 (33h10m43.402944544s) [upload=29117]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 20:51:11.959151951 (33h10m43.655000704s) [seg_pos={0 0}, upload=29117]: Begin upload segment... Oct 31 20:51:11.959219510 (33h10m43.655068263s) [seg_pos={0 0}, upload=29117]: Done begin upload segment. Oct 31 20:51:11.959257482 (33h10m43.655106235s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959352193 (33h10m43.655200936s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959984507 (33h10m43.65583325s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960349932 (33h10m43.656198685s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960434654 (33h10m43.656283397s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960753571 (33h10m43.656602314s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960782867 (33h10m43.65663162s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961093809 (33h10m43.656942562s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961223666 (33h10m43.657072409s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961396354 (33h10m43.657245097s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961606384 (33h10m43.657455127s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961727265 (33h10m43.657576008s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962038738 (33h10m43.657887481s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962095516 (33h10m43.657944259s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962430774 (33h10m43.658279517s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962491260 (33h10m43.658340013s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:13.589873545 (33h10m45.285722288s) [upload=29117]: Next returned nil segment Oct 31 20:51:13.589889816 (33h10m45.285738569s) [upload=29117]: Waiting for error group managing segments... Oct 31 20:51:13.973997709 (33h10m45.669846462s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.977395970 (33h10m45.673244713s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.979447918 (33h10m45.675296671s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.980901436 (33h10m45.676750189s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.986457284 (33h10m45.682306088s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.004274321 (33h10m45.700123084s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.012149448 (33h10m45.707998201s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.027225746 (33h10m45.723074499s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.043308651 (33h10m45.739157404s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.050118539 (33h10m45.745967292s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.065628894 (33h10m45.761477637s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.079898897 (33h10m45.77574765s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:41.002915508 (33h11m12.698764281s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection Oct 31 20:51:41.003131980 (33h11m12.698980723s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection Oct 31 20:51:41.003350967 (33h11m12.69919973s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection Oct 31 20:51:41.003851250 (33h11m12.699700003s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: read tcp 188.166.225.170:49810->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:51:41.003903128 (33h11m12.699751891s) [seg_pos={0 0}, upload=29117]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 20:51:41.248997100 (33h11m12.944845853s) [upload=29117]: Done uploading segments... Oct 31 20:51:41.249037957 (33h11m12.944886711s) [upload=29117]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.437805099s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................X....X] duration: 30.001517325s trace: https://jaeger.dev.storj.io/trace/35569b98f076d00a FAIL (2025-10-31 20:51:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001541151s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................................................XX....X] duration: 30.247927847s trace: https://jaeger.dev.storj.io/trace/61feae49d47bc60e FAIL (2025-10-31 20:51:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 20:51:11.706455020 (33h10m43.402303773s) [upload=29117]: Starting upload Oct 31 20:51:11.706524262 (33h10m43.402373005s) [upload=29117]: Done starting upload Oct 31 20:51:11.706833080 (33h10m43.402681823s) [upload=29117]: Uploading segments... Oct 31 20:51:11.707027190 (33h10m43.402875943s) [upload=29117]: Got next segment. Inline: false Oct 31 20:51:11.707074009 (33h10m43.402922762s) [upload=29117]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 20:51:11.707095790 (33h10m43.402944544s) [upload=29117]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 20:51:11.959151951 (33h10m43.655000704s) [seg_pos={0 0}, upload=29117]: Begin upload segment... Oct 31 20:51:11.959219510 (33h10m43.655068263s) [seg_pos={0 0}, upload=29117]: Done begin upload segment. Oct 31 20:51:11.959257482 (33h10m43.655106235s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959352193 (33h10m43.655200936s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.959984507 (33h10m43.65583325s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960349932 (33h10m43.656198685s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960434654 (33h10m43.656283397s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960753571 (33h10m43.656602314s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.960782867 (33h10m43.65663162s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961093809 (33h10m43.656942562s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961223666 (33h10m43.657072409s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961396354 (33h10m43.657245097s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961606384 (33h10m43.657455127s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.961727265 (33h10m43.657576008s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962038738 (33h10m43.657887481s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962095516 (33h10m43.657944259s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962430774 (33h10m43.658279517s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:11.962491260 (33h10m43.658340013s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Uploading piece... Oct 31 20:51:13.589873545 (33h10m45.285722288s) [upload=29117]: Next returned nil segment Oct 31 20:51:13.589889816 (33h10m45.285738569s) [upload=29117]: Waiting for error group managing segments... Oct 31 20:51:13.973997709 (33h10m45.669846462s) [noise=true , address=145.79.227.9:4366 , piece_id=B3MDET2NK75LH2PFOTQQQKMTOORYPKBIPYMVDMRLAPALMFHMMTUQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.977395970 (33h10m45.673244713s) [noise=true , address=145.79.227.8:4331 , piece_id=WBJEFDSQWI57LUVOFU46SW55X5A4TMAQDQYQUJBSL7PUK76BEWFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.979447918 (33h10m45.675296671s) [noise=true , address=145.79.227.14:4840 , piece_id=NSQMPI3HHFYPKRTICMDIAKLP4OYYXK3PUKWAQWGDUKXIMMAET26Q, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.980901436 (33h10m45.676750189s) [noise=true , address=145.79.227.31:4797 , piece_id=RTY2YTW7PA34HHO6NTN7WDGFUZTU5SF3B2TPDH76Z5HLZRRPRXUA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:13.986457284 (33h10m45.682306088s) [noise=true , address=145.79.227.24:4581 , piece_id=5QMNPMLJAQXITYKVAMORL5C2ETAVDLAGYUFMFLXI4SJ3HK67LXKA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.004274321 (33h10m45.700123084s) [noise=true , address=145.79.227.13:4803 , piece_id=U3TRIALHSTLPGF44OTFLLTHPYHM2LU57DYF2ONNQSPFZSFJT3JGA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.012149448 (33h10m45.707998201s) [noise=true , address=145.79.227.4:4185 , piece_id=F7WTGPH6LJR7KKPKLEITSPS24KRXGBYAPVBH2A7RVCPCQZQKVKFA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.027225746 (33h10m45.723074499s) [noise=true , address=145.79.227.32:4836 , piece_id=PYCFWHO5ACF33RG5CXXTUURHU5D3XHXFH7GWQXGCULB3LQCABCRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.043308651 (33h10m45.739157404s) [noise=true , address=145.79.227.21:4475 , piece_id=X47C7EP73DAXLRBDFDNLYLBRVTUJ7E25D6HE2BKNNFJCYWE2GETA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.050118539 (33h10m45.745967292s) [noise=true , address=145.79.227.23:4546 , piece_id=XUIN6VBEE64LCSA5UNYXSN3BR74IL6D5VVEPA2KELRN4HYFRPIIA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.065628894 (33h10m45.761477637s) [noise=true , address=145.79.227.28:4725 , piece_id=X2NUNP6LLBGDB2B2ZIT5MBW77PFJ3AZHJEPUDW3JWZ4CXTYXN6PQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:14.079898897 (33h10m45.77574765s) [noise=true , address=145.79.227.31:4799 , piece_id=M2TGJ7FO5UPK5F32KJWOB4J2YAWF5QYB2MLI3CSLVZQXLSOLPOZQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: <nil> Oct 31 20:51:41.002915508 (33h11m12.698764281s) [noise=true , address=77.90.31.12:4592 , piece_id=PW52BFRZZDXQWEXJUBDYNPSU6QGDV7MEJKUM32XKHJIQPQUHAH7A, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47064->77.90.31.12:4592: use of closed network connection Oct 31 20:51:41.003131980 (33h11m12.698980723s) [noise=true , address=77.90.31.10:4519 , piece_id=XSDLPRAYDE7OPRTJBYSMM3YZNGEPLVAPN5YYKBJNBDGXALNM2ISA, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37294->77.90.31.10:4519: use of closed network connection Oct 31 20:51:41.003350967 (33h11m12.69919973s) [noise=true , address=77.90.31.13:4626 , piece_id=LP26J4GFG7AMZALJX6UVH4RBYCLWESEDNVF2ZJSNLEDYOFCSDSIQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, address:77.90.31.13:4626): protocol: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59286->77.90.31.13:4626: use of closed network connection Oct 31 20:51:41.003851250 (33h11m12.699700003s) [noise=true , address=77.90.31.18:4807 , piece_id=7GNYW2VEL4YS5PTCRGICZEVNDGLVIBNATAU6DGLMLQWVBHAVJQRQ, seg_pos={0 0}, upload=29117]: Done uploading piece. err: ecclient: upload failed (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS, address:77.90.31.18:4807): protocol: read tcp 188.166.225.170:49810->77.90.31.18:4807: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:51:41.003903128 (33h11m12.699751891s) [seg_pos={0 0}, upload=29117]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 20:51:41.248997100 (33h11m12.944845853s) [upload=29117]: Done uploading segments... Oct 31 20:51:41.249037957 (33h11m12.944886711s) [upload=29117]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.247954388s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 337.39868ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/48774610bbd09e70 FAIL (2025-10-31 20:51:34 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 337.42454ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................XX....] duration: 5.822787965s trace: https://jaeger.dev.storj.io/trace/57ef7f61138e7c35 FAIL (2025-10-31 20:50:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 20:47:42.508656855 (33h7m14.204505608s) [upload=29065]: Starting upload Oct 31 20:47:42.508711358 (33h7m14.204560111s) [upload=29065]: Done starting upload Oct 31 20:47:42.508835555 (33h7m14.204684298s) [upload=29065]: Uploading segments... Oct 31 20:47:42.508919635 (33h7m14.204768378s) [upload=29065]: Got next segment. Inline: false Oct 31 20:47:42.508957166 (33h7m14.204805919s) [upload=29065]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 20:47:42.508981583 (33h7m14.204830336s) [upload=29065]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 20:47:42.757470730 (33h7m14.453319493s) [seg_pos={0 0}, upload=29065]: Begin upload segment... Oct 31 20:47:42.757527728 (33h7m14.453376471s) [seg_pos={0 0}, upload=29065]: Done begin upload segment. Oct 31 20:47:42.757564037 (33h7m14.45341278s) [noise=true , address=77.90.31.17:4772 , piece_id=ZTNTL3LMNN6OTUSENUCIJ6QDPD7VDELTXJ634YAW5VJC6CE4EUDQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.758551809 (33h7m14.454400572s) [noise=true , address=45.91.116.8:4045 , piece_id=RBLYSRSYCLW4BM42HMCPWPUTIPZJTPOUBOFST2ZGTDRYRCMYPG4Q, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.758883590 (33h7m14.454732353s) [noise=true , address=145.79.227.13:4802 , piece_id=EX5NKH5RHPEXCEF2S3KYZ6DRMIL4AF5IIC6EQHLOVIUNVVCM3FWA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759297398 (33h7m14.455146151s) [noise=true , address=145.79.227.10:4403 , piece_id=RIMPQPW53W2ZFWPUC2C6PHEAO4KYXOHEXYVP4OC5XSLKWARUUG5A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759395595 (33h7m14.455244338s) [noise=true , address=77.90.31.14:4661 , piece_id=FWKOERG4NXKSOIWJVFEJNV5XXOKRVTHM7DZFPDAYRK3T6WKFENHQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759665890 (33h7m14.455514643s) [noise=true , address=145.79.227.20:4438 , piece_id=KULG2SULBLTY3OFV5JGPQPPB647MA6G7GR543W4ITIWDTVWS5RNA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759925394 (33h7m14.455774147s) [noise=true , address=77.90.31.16:4734 , piece_id=AJNH3MWGRYCYRVSYNKAR5OUWVP3QAFHCR4A4VUXHGIUQEYGLL52A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760006399 (33h7m14.455855142s) [noise=true , address=77.90.31.17:4770 , piece_id=ROS5JGE3AKSEPE74IA2VLZKLZ4SUZUVNNRS5G63RDTMBGOXUU6LQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760364751 (33h7m14.456213514s) [noise=true , address=145.79.227.23:4546 , piece_id=73GYIV6FEN4GTBGNIXEO6NHFVBLWFEAKDW5MSHWM3PZVKUBEE63Q, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760496121 (33h7m14.456344874s) [noise=true , address=145.79.227.4:4185 , piece_id=PHN2D46K6N7EXQSFLLC653IXIPQ5KQURPGRKB5VIO2HQAGAHYS3A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760759412 (33h7m14.456608165s) [noise=true , address=145.79.227.8:4331 , piece_id=QD7MLX5QFRR7J5Z4AIOPFHU77SWSVWHW676F3IBPAU2BBGF6CQTQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760992276 (33h7m14.456841019s) [noise=true , address=145.79.227.24:4581 , piece_id=LX2A6M3I5YY4U2YDLFIFJ7RJA3QSFNHKWTW2SSRFQWKMVMSV55CA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.761119158 (33h7m14.456967911s) [noise=true , address=145.79.227.32:4835 , piece_id=YCG73FQ45UKT6XSC3YSN6R6CK2N25C34OV4LQ3PDGSKY7D5M334A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.761488600 (33h7m14.457337353s) [noise=true , address=145.79.227.17:4971 , piece_id=5VDD2RS3IHKDCMZYGEPCHMMAZMJ7G5LYIXLRQTC4KBKJDQ2MIH3Q, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.761536843 (33h7m14.457385586s) [noise=true , address=145.79.227.21:4474 , piece_id=APCGJFDNPWM4BJEEHP7EWQHQPRIHGW4NO7TMCGU32ELTRBF22LGA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.762222498 (33h7m14.458071241s) [noise=true , address=145.79.227.9:4366 , piece_id=I7GGALAWWWLA5HXX2EOPJCTIVZXI3YGLNE2K2R4ISPLU5MVXJNLA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:44.473460063 (33h7m16.169308816s) [upload=29065]: Next returned nil segment Oct 31 20:47:44.473587325 (33h7m16.169436088s) [upload=29065]: Waiting for error group managing segments... Oct 31 20:47:44.757605346 (33h7m16.453454089s) [noise=true , address=145.79.227.9:4366 , piece_id=I7GGALAWWWLA5HXX2EOPJCTIVZXI3YGLNE2K2R4ISPLU5MVXJNLA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.764557314 (33h7m16.460406067s) [noise=true , address=145.79.227.32:4835 , piece_id=YCG73FQ45UKT6XSC3YSN6R6CK2N25C34OV4LQ3PDGSKY7D5M334A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.774899719 (33h7m16.470748462s) [noise=true , address=145.79.227.4:4185 , piece_id=PHN2D46K6N7EXQSFLLC653IXIPQ5KQURPGRKB5VIO2HQAGAHYS3A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.775470495 (33h7m16.471319238s) [noise=true , address=145.79.227.10:4403 , piece_id=RIMPQPW53W2ZFWPUC2C6PHEAO4KYXOHEXYVP4OC5XSLKWARUUG5A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.794178068 (33h7m16.490026821s) [noise=true , address=145.79.227.24:4581 , piece_id=LX2A6M3I5YY4U2YDLFIFJ7RJA3QSFNHKWTW2SSRFQWKMVMSV55CA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.797767073 (33h7m16.493615816s) [noise=true , address=145.79.227.23:4546 , piece_id=73GYIV6FEN4GTBGNIXEO6NHFVBLWFEAKDW5MSHWM3PZVKUBEE63Q, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.809232004 (33h7m16.505080757s) [noise=true , address=145.79.227.8:4331 , piece_id=QD7MLX5QFRR7J5Z4AIOPFHU77SWSVWHW676F3IBPAU2BBGF6CQTQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.819266102 (33h7m16.515114855s) [noise=true , address=145.79.227.20:4438 , piece_id=KULG2SULBLTY3OFV5JGPQPPB647MA6G7GR543W4ITIWDTVWS5RNA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.828970591 (33h7m16.524819344s) [noise=true , address=145.79.227.13:4802 , piece_id=EX5NKH5RHPEXCEF2S3KYZ6DRMIL4AF5IIC6EQHLOVIUNVVCM3FWA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.841564723 (33h7m16.537413476s) [noise=true , address=145.79.227.17:4971 , piece_id=5VDD2RS3IHKDCMZYGEPCHMMAZMJ7G5LYIXLRQTC4KBKJDQ2MIH3Q, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.862610308 (33h7m16.558459051s) [noise=true , address=145.79.227.21:4474 , piece_id=APCGJFDNPWM4BJEEHP7EWQHQPRIHGW4NO7TMCGU32ELTRBF22LGA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:48:10.667054932 (33h7m42.362903695s) [noise=true , address=77.90.31.14:4661 , piece_id=FWKOERG4NXKSOIWJVFEJNV5XXOKRVTHM7DZFPDAYRK3T6WKFENHQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:48:11.815498164 (33h7m43.511346907s) [noise=true , address=45.91.116.8:4045 , piece_id=RBLYSRSYCLW4BM42HMCPWPUTIPZJTPOUBOFST2ZGTDRYRCMYPG4Q, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 188.166.225.170:49100->45.91.116.8:4045: use of closed network connection; write tcp 188.166.225.170:49100->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49100->45.91.116.8:4045: use of closed network connection Oct 31 20:48:11.815924096 (33h7m43.511772849s) [noise=true , address=77.90.31.17:4772 , piece_id=ZTNTL3LMNN6OTUSENUCIJ6QDPD7VDELTXJ634YAW5VJC6CE4EUDQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42670->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:48:11.816087947 (33h7m43.5119367s) [noise=true , address=77.90.31.16:4734 , piece_id=AJNH3MWGRYCYRVSYNKAR5OUWVP3QAFHCR4A4VUXHGIUQEYGLL52A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 188.166.225.170:55794->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:48:11.816286556 (33h7m43.512135309s) [noise=true , address=77.90.31.17:4770 , piece_id=ROS5JGE3AKSEPE74IA2VLZKLZ4SUZUVNNRS5G63RDTMBGOXUU6LQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 188.166.225.170:55648->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:48:11.816319889 (33h7m43.512168642s) [seg_pos={0 0}, upload=29065]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 20:48:12.061999186 (33h7m43.757847949s) [upload=29065]: Done uploading segments... Oct 31 20:48:12.062045554 (33h7m43.757894317s) [upload=29065]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.822812212s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.058471021s trace: https://jaeger.dev.storj.io/trace/5bed7337abd66ce6 FAIL (2025-10-31 20:50:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.058501579s edge gateway us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 322.063547ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/48774610bbd09e70 FAIL (2025-10-31 20:50:34 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 322.096131ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................XX...] duration: 5.981092907s trace: https://jaeger.dev.storj.io/trace/57ef7f61138e7c35 FAIL (2025-10-31 20:50:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 20:47:42.508656855 (33h7m14.204505608s) [upload=29065]: Starting upload Oct 31 20:47:42.508711358 (33h7m14.204560111s) [upload=29065]: Done starting upload Oct 31 20:47:42.508835555 (33h7m14.204684298s) [upload=29065]: Uploading segments... Oct 31 20:47:42.508919635 (33h7m14.204768378s) [upload=29065]: Got next segment. Inline: false Oct 31 20:47:42.508957166 (33h7m14.204805919s) [upload=29065]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 20:47:42.508981583 (33h7m14.204830336s) [upload=29065]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 20:47:42.757470730 (33h7m14.453319493s) [seg_pos={0 0}, upload=29065]: Begin upload segment... Oct 31 20:47:42.757527728 (33h7m14.453376471s) [seg_pos={0 0}, upload=29065]: Done begin upload segment. Oct 31 20:47:42.757564037 (33h7m14.45341278s) [noise=true , address=77.90.31.17:4772 , piece_id=ZTNTL3LMNN6OTUSENUCIJ6QDPD7VDELTXJ634YAW5VJC6CE4EUDQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.758551809 (33h7m14.454400572s) [noise=true , address=45.91.116.8:4045 , piece_id=RBLYSRSYCLW4BM42HMCPWPUTIPZJTPOUBOFST2ZGTDRYRCMYPG4Q, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.758883590 (33h7m14.454732353s) [noise=true , address=145.79.227.13:4802 , piece_id=EX5NKH5RHPEXCEF2S3KYZ6DRMIL4AF5IIC6EQHLOVIUNVVCM3FWA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759297398 (33h7m14.455146151s) [noise=true , address=145.79.227.10:4403 , piece_id=RIMPQPW53W2ZFWPUC2C6PHEAO4KYXOHEXYVP4OC5XSLKWARUUG5A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759395595 (33h7m14.455244338s) [noise=true , address=77.90.31.14:4661 , piece_id=FWKOERG4NXKSOIWJVFEJNV5XXOKRVTHM7DZFPDAYRK3T6WKFENHQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759665890 (33h7m14.455514643s) [noise=true , address=145.79.227.20:4438 , piece_id=KULG2SULBLTY3OFV5JGPQPPB647MA6G7GR543W4ITIWDTVWS5RNA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759925394 (33h7m14.455774147s) [noise=true , address=77.90.31.16:4734 , piece_id=AJNH3MWGRYCYRVSYNKAR5OUWVP3QAFHCR4A4VUXHGIUQEYGLL52A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760006399 (33h7m14.455855142s) [noise=true , address=77.90.31.17:4770 , piece_id=ROS5JGE3AKSEPE74IA2VLZKLZ4SUZUVNNRS5G63RDTMBGOXUU6LQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760364751 (33h7m14.456213514s) [noise=true , address=145.79.227.23:4546 , piece_id=73GYIV6FEN4GTBGNIXEO6NHFVBLWFEAKDW5MSHWM3PZVKUBEE63Q, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760496121 (33h7m14.456344874s) [noise=true , address=145.79.227.4:4185 , piece_id=PHN2D46K6N7EXQSFLLC653IXIPQ5KQURPGRKB5VIO2HQAGAHYS3A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760759412 (33h7m14.456608165s) [noise=true , address=145.79.227.8:4331 , piece_id=QD7MLX5QFRR7J5Z4AIOPFHU77SWSVWHW676F3IBPAU2BBGF6CQTQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760992276 (33h7m14.456841019s) [noise=true , address=145.79.227.24:4581 , piece_id=LX2A6M3I5YY4U2YDLFIFJ7RJA3QSFNHKWTW2SSRFQWKMVMSV55CA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.761119158 (33h7m14.456967911s) [noise=true , address=145.79.227.32:4835 , piece_id=YCG73FQ45UKT6XSC3YSN6R6CK2N25C34OV4LQ3PDGSKY7D5M334A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.761488600 (33h7m14.457337353s) [noise=true , address=145.79.227.17:4971 , piece_id=5VDD2RS3IHKDCMZYGEPCHMMAZMJ7G5LYIXLRQTC4KBKJDQ2MIH3Q, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.761536843 (33h7m14.457385586s) [noise=true , address=145.79.227.21:4474 , piece_id=APCGJFDNPWM4BJEEHP7EWQHQPRIHGW4NO7TMCGU32ELTRBF22LGA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.762222498 (33h7m14.458071241s) [noise=true , address=145.79.227.9:4366 , piece_id=I7GGALAWWWLA5HXX2EOPJCTIVZXI3YGLNE2K2R4ISPLU5MVXJNLA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:44.473460063 (33h7m16.169308816s) [upload=29065]: Next returned nil segment Oct 31 20:47:44.473587325 (33h7m16.169436088s) [upload=29065]: Waiting for error group managing segments... Oct 31 20:47:44.757605346 (33h7m16.453454089s) [noise=true , address=145.79.227.9:4366 , piece_id=I7GGALAWWWLA5HXX2EOPJCTIVZXI3YGLNE2K2R4ISPLU5MVXJNLA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.764557314 (33h7m16.460406067s) [noise=true , address=145.79.227.32:4835 , piece_id=YCG73FQ45UKT6XSC3YSN6R6CK2N25C34OV4LQ3PDGSKY7D5M334A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.774899719 (33h7m16.470748462s) [noise=true , address=145.79.227.4:4185 , piece_id=PHN2D46K6N7EXQSFLLC653IXIPQ5KQURPGRKB5VIO2HQAGAHYS3A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.775470495 (33h7m16.471319238s) [noise=true , address=145.79.227.10:4403 , piece_id=RIMPQPW53W2ZFWPUC2C6PHEAO4KYXOHEXYVP4OC5XSLKWARUUG5A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.794178068 (33h7m16.490026821s) [noise=true , address=145.79.227.24:4581 , piece_id=LX2A6M3I5YY4U2YDLFIFJ7RJA3QSFNHKWTW2SSRFQWKMVMSV55CA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.797767073 (33h7m16.493615816s) [noise=true , address=145.79.227.23:4546 , piece_id=73GYIV6FEN4GTBGNIXEO6NHFVBLWFEAKDW5MSHWM3PZVKUBEE63Q, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.809232004 (33h7m16.505080757s) [noise=true , address=145.79.227.8:4331 , piece_id=QD7MLX5QFRR7J5Z4AIOPFHU77SWSVWHW676F3IBPAU2BBGF6CQTQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.819266102 (33h7m16.515114855s) [noise=true , address=145.79.227.20:4438 , piece_id=KULG2SULBLTY3OFV5JGPQPPB647MA6G7GR543W4ITIWDTVWS5RNA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.828970591 (33h7m16.524819344s) [noise=true , address=145.79.227.13:4802 , piece_id=EX5NKH5RHPEXCEF2S3KYZ6DRMIL4AF5IIC6EQHLOVIUNVVCM3FWA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.841564723 (33h7m16.537413476s) [noise=true , address=145.79.227.17:4971 , piece_id=5VDD2RS3IHKDCMZYGEPCHMMAZMJ7G5LYIXLRQTC4KBKJDQ2MIH3Q, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.862610308 (33h7m16.558459051s) [noise=true , address=145.79.227.21:4474 , piece_id=APCGJFDNPWM4BJEEHP7EWQHQPRIHGW4NO7TMCGU32ELTRBF22LGA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:48:10.667054932 (33h7m42.362903695s) [noise=true , address=77.90.31.14:4661 , piece_id=FWKOERG4NXKSOIWJVFEJNV5XXOKRVTHM7DZFPDAYRK3T6WKFENHQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:48:11.815498164 (33h7m43.511346907s) [noise=true , address=45.91.116.8:4045 , piece_id=RBLYSRSYCLW4BM42HMCPWPUTIPZJTPOUBOFST2ZGTDRYRCMYPG4Q, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 188.166.225.170:49100->45.91.116.8:4045: use of closed network connection; write tcp 188.166.225.170:49100->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49100->45.91.116.8:4045: use of closed network connection Oct 31 20:48:11.815924096 (33h7m43.511772849s) [noise=true , address=77.90.31.17:4772 , piece_id=ZTNTL3LMNN6OTUSENUCIJ6QDPD7VDELTXJ634YAW5VJC6CE4EUDQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42670->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:48:11.816087947 (33h7m43.5119367s) [noise=true , address=77.90.31.16:4734 , piece_id=AJNH3MWGRYCYRVSYNKAR5OUWVP3QAFHCR4A4VUXHGIUQEYGLL52A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 188.166.225.170:55794->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:48:11.816286556 (33h7m43.512135309s) [noise=true , address=77.90.31.17:4770 , piece_id=ROS5JGE3AKSEPE74IA2VLZKLZ4SUZUVNNRS5G63RDTMBGOXUU6LQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 188.166.225.170:55648->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:48:11.816319889 (33h7m43.512168642s) [seg_pos={0 0}, upload=29065]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 20:48:12.061999186 (33h7m43.757847949s) [upload=29065]: Done uploading segments... Oct 31 20:48:12.062045554 (33h7m43.757894317s) [upload=29065]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.981125218s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 301.021481ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/48774610bbd09e70 FAIL (2025-10-31 20:50:10 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 301.051216ms core website
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.056699297s trace: https://jaeger.dev.storj.io/trace/5bed7337abd66ce6 FAIL (2025-10-31 20:49:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.056740016s edge gateway us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 324.150181ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/48774610bbd09e70 FAIL (2025-10-31 20:49:43 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 324.18576ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 5.872055332s trace: https://jaeger.dev.storj.io/trace/57ef7f61138e7c35 FAIL (2025-10-31 20:49:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 20:47:42.508656855 (33h7m14.204505608s) [upload=29065]: Starting upload Oct 31 20:47:42.508711358 (33h7m14.204560111s) [upload=29065]: Done starting upload Oct 31 20:47:42.508835555 (33h7m14.204684298s) [upload=29065]: Uploading segments... Oct 31 20:47:42.508919635 (33h7m14.204768378s) [upload=29065]: Got next segment. Inline: false Oct 31 20:47:42.508957166 (33h7m14.204805919s) [upload=29065]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 20:47:42.508981583 (33h7m14.204830336s) [upload=29065]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 20:47:42.757470730 (33h7m14.453319493s) [seg_pos={0 0}, upload=29065]: Begin upload segment... Oct 31 20:47:42.757527728 (33h7m14.453376471s) [seg_pos={0 0}, upload=29065]: Done begin upload segment. Oct 31 20:47:42.757564037 (33h7m14.45341278s) [noise=true , address=77.90.31.17:4772 , piece_id=ZTNTL3LMNN6OTUSENUCIJ6QDPD7VDELTXJ634YAW5VJC6CE4EUDQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.758551809 (33h7m14.454400572s) [noise=true , address=45.91.116.8:4045 , piece_id=RBLYSRSYCLW4BM42HMCPWPUTIPZJTPOUBOFST2ZGTDRYRCMYPG4Q, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.758883590 (33h7m14.454732353s) [noise=true , address=145.79.227.13:4802 , piece_id=EX5NKH5RHPEXCEF2S3KYZ6DRMIL4AF5IIC6EQHLOVIUNVVCM3FWA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759297398 (33h7m14.455146151s) [noise=true , address=145.79.227.10:4403 , piece_id=RIMPQPW53W2ZFWPUC2C6PHEAO4KYXOHEXYVP4OC5XSLKWARUUG5A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759395595 (33h7m14.455244338s) [noise=true , address=77.90.31.14:4661 , piece_id=FWKOERG4NXKSOIWJVFEJNV5XXOKRVTHM7DZFPDAYRK3T6WKFENHQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759665890 (33h7m14.455514643s) [noise=true , address=145.79.227.20:4438 , piece_id=KULG2SULBLTY3OFV5JGPQPPB647MA6G7GR543W4ITIWDTVWS5RNA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759925394 (33h7m14.455774147s) [noise=true , address=77.90.31.16:4734 , piece_id=AJNH3MWGRYCYRVSYNKAR5OUWVP3QAFHCR4A4VUXHGIUQEYGLL52A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760006399 (33h7m14.455855142s) [noise=true , address=77.90.31.17:4770 , piece_id=ROS5JGE3AKSEPE74IA2VLZKLZ4SUZUVNNRS5G63RDTMBGOXUU6LQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760364751 (33h7m14.456213514s) [noise=true , address=145.79.227.23:4546 , piece_id=73GYIV6FEN4GTBGNIXEO6NHFVBLWFEAKDW5MSHWM3PZVKUBEE63Q, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760496121 (33h7m14.456344874s) [noise=true , address=145.79.227.4:4185 , piece_id=PHN2D46K6N7EXQSFLLC653IXIPQ5KQURPGRKB5VIO2HQAGAHYS3A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760759412 (33h7m14.456608165s) [noise=true , address=145.79.227.8:4331 , piece_id=QD7MLX5QFRR7J5Z4AIOPFHU77SWSVWHW676F3IBPAU2BBGF6CQTQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760992276 (33h7m14.456841019s) [noise=true , address=145.79.227.24:4581 , piece_id=LX2A6M3I5YY4U2YDLFIFJ7RJA3QSFNHKWTW2SSRFQWKMVMSV55CA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.761119158 (33h7m14.456967911s) [noise=true , address=145.79.227.32:4835 , piece_id=YCG73FQ45UKT6XSC3YSN6R6CK2N25C34OV4LQ3PDGSKY7D5M334A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.761488600 (33h7m14.457337353s) [noise=true , address=145.79.227.17:4971 , piece_id=5VDD2RS3IHKDCMZYGEPCHMMAZMJ7G5LYIXLRQTC4KBKJDQ2MIH3Q, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.761536843 (33h7m14.457385586s) [noise=true , address=145.79.227.21:4474 , piece_id=APCGJFDNPWM4BJEEHP7EWQHQPRIHGW4NO7TMCGU32ELTRBF22LGA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.762222498 (33h7m14.458071241s) [noise=true , address=145.79.227.9:4366 , piece_id=I7GGALAWWWLA5HXX2EOPJCTIVZXI3YGLNE2K2R4ISPLU5MVXJNLA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:44.473460063 (33h7m16.169308816s) [upload=29065]: Next returned nil segment Oct 31 20:47:44.473587325 (33h7m16.169436088s) [upload=29065]: Waiting for error group managing segments... Oct 31 20:47:44.757605346 (33h7m16.453454089s) [noise=true , address=145.79.227.9:4366 , piece_id=I7GGALAWWWLA5HXX2EOPJCTIVZXI3YGLNE2K2R4ISPLU5MVXJNLA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.764557314 (33h7m16.460406067s) [noise=true , address=145.79.227.32:4835 , piece_id=YCG73FQ45UKT6XSC3YSN6R6CK2N25C34OV4LQ3PDGSKY7D5M334A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.774899719 (33h7m16.470748462s) [noise=true , address=145.79.227.4:4185 , piece_id=PHN2D46K6N7EXQSFLLC653IXIPQ5KQURPGRKB5VIO2HQAGAHYS3A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.775470495 (33h7m16.471319238s) [noise=true , address=145.79.227.10:4403 , piece_id=RIMPQPW53W2ZFWPUC2C6PHEAO4KYXOHEXYVP4OC5XSLKWARUUG5A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.794178068 (33h7m16.490026821s) [noise=true , address=145.79.227.24:4581 , piece_id=LX2A6M3I5YY4U2YDLFIFJ7RJA3QSFNHKWTW2SSRFQWKMVMSV55CA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.797767073 (33h7m16.493615816s) [noise=true , address=145.79.227.23:4546 , piece_id=73GYIV6FEN4GTBGNIXEO6NHFVBLWFEAKDW5MSHWM3PZVKUBEE63Q, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.809232004 (33h7m16.505080757s) [noise=true , address=145.79.227.8:4331 , piece_id=QD7MLX5QFRR7J5Z4AIOPFHU77SWSVWHW676F3IBPAU2BBGF6CQTQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.819266102 (33h7m16.515114855s) [noise=true , address=145.79.227.20:4438 , piece_id=KULG2SULBLTY3OFV5JGPQPPB647MA6G7GR543W4ITIWDTVWS5RNA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.828970591 (33h7m16.524819344s) [noise=true , address=145.79.227.13:4802 , piece_id=EX5NKH5RHPEXCEF2S3KYZ6DRMIL4AF5IIC6EQHLOVIUNVVCM3FWA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.841564723 (33h7m16.537413476s) [noise=true , address=145.79.227.17:4971 , piece_id=5VDD2RS3IHKDCMZYGEPCHMMAZMJ7G5LYIXLRQTC4KBKJDQ2MIH3Q, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.862610308 (33h7m16.558459051s) [noise=true , address=145.79.227.21:4474 , piece_id=APCGJFDNPWM4BJEEHP7EWQHQPRIHGW4NO7TMCGU32ELTRBF22LGA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:48:10.667054932 (33h7m42.362903695s) [noise=true , address=77.90.31.14:4661 , piece_id=FWKOERG4NXKSOIWJVFEJNV5XXOKRVTHM7DZFPDAYRK3T6WKFENHQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:48:11.815498164 (33h7m43.511346907s) [noise=true , address=45.91.116.8:4045 , piece_id=RBLYSRSYCLW4BM42HMCPWPUTIPZJTPOUBOFST2ZGTDRYRCMYPG4Q, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 188.166.225.170:49100->45.91.116.8:4045: use of closed network connection; write tcp 188.166.225.170:49100->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49100->45.91.116.8:4045: use of closed network connection Oct 31 20:48:11.815924096 (33h7m43.511772849s) [noise=true , address=77.90.31.17:4772 , piece_id=ZTNTL3LMNN6OTUSENUCIJ6QDPD7VDELTXJ634YAW5VJC6CE4EUDQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42670->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:48:11.816087947 (33h7m43.5119367s) [noise=true , address=77.90.31.16:4734 , piece_id=AJNH3MWGRYCYRVSYNKAR5OUWVP3QAFHCR4A4VUXHGIUQEYGLL52A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 188.166.225.170:55794->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:48:11.816286556 (33h7m43.512135309s) [noise=true , address=77.90.31.17:4770 , piece_id=ROS5JGE3AKSEPE74IA2VLZKLZ4SUZUVNNRS5G63RDTMBGOXUU6LQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 188.166.225.170:55648->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:48:11.816319889 (33h7m43.512168642s) [seg_pos={0 0}, upload=29065]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 20:48:12.061999186 (33h7m43.757847949s) [upload=29065]: Done uploading segments... Oct 31 20:48:12.062045554 (33h7m43.757894317s) [upload=29065]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.872097963s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.171810198s trace: https://jaeger.dev.storj.io/trace/5bed7337abd66ce6 FAIL (2025-10-31 20:49:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.171839304s edge gateway us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.............X.] duration: 314.822416ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/48774610bbd09e70 FAIL (2025-10-31 20:49:10 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 314.846683ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..............X] duration: 218.582157ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/48774610bbd09e70 FAIL (2025-10-31 20:48:38 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:38346->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 218.606684ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 6.055824017s trace: https://jaeger.dev.storj.io/trace/57ef7f61138e7c35 FAIL (2025-10-31 20:48:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 20:47:42.508656855 (33h7m14.204505608s) [upload=29065]: Starting upload Oct 31 20:47:42.508711358 (33h7m14.204560111s) [upload=29065]: Done starting upload Oct 31 20:47:42.508835555 (33h7m14.204684298s) [upload=29065]: Uploading segments... Oct 31 20:47:42.508919635 (33h7m14.204768378s) [upload=29065]: Got next segment. Inline: false Oct 31 20:47:42.508957166 (33h7m14.204805919s) [upload=29065]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 20:47:42.508981583 (33h7m14.204830336s) [upload=29065]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 20:47:42.757470730 (33h7m14.453319493s) [seg_pos={0 0}, upload=29065]: Begin upload segment... Oct 31 20:47:42.757527728 (33h7m14.453376471s) [seg_pos={0 0}, upload=29065]: Done begin upload segment. Oct 31 20:47:42.757564037 (33h7m14.45341278s) [noise=true , address=77.90.31.17:4772 , piece_id=ZTNTL3LMNN6OTUSENUCIJ6QDPD7VDELTXJ634YAW5VJC6CE4EUDQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.758551809 (33h7m14.454400572s) [noise=true , address=45.91.116.8:4045 , piece_id=RBLYSRSYCLW4BM42HMCPWPUTIPZJTPOUBOFST2ZGTDRYRCMYPG4Q, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.758883590 (33h7m14.454732353s) [noise=true , address=145.79.227.13:4802 , piece_id=EX5NKH5RHPEXCEF2S3KYZ6DRMIL4AF5IIC6EQHLOVIUNVVCM3FWA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759297398 (33h7m14.455146151s) [noise=true , address=145.79.227.10:4403 , piece_id=RIMPQPW53W2ZFWPUC2C6PHEAO4KYXOHEXYVP4OC5XSLKWARUUG5A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759395595 (33h7m14.455244338s) [noise=true , address=77.90.31.14:4661 , piece_id=FWKOERG4NXKSOIWJVFEJNV5XXOKRVTHM7DZFPDAYRK3T6WKFENHQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759665890 (33h7m14.455514643s) [noise=true , address=145.79.227.20:4438 , piece_id=KULG2SULBLTY3OFV5JGPQPPB647MA6G7GR543W4ITIWDTVWS5RNA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759925394 (33h7m14.455774147s) [noise=true , address=77.90.31.16:4734 , piece_id=AJNH3MWGRYCYRVSYNKAR5OUWVP3QAFHCR4A4VUXHGIUQEYGLL52A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760006399 (33h7m14.455855142s) [noise=true , address=77.90.31.17:4770 , piece_id=ROS5JGE3AKSEPE74IA2VLZKLZ4SUZUVNNRS5G63RDTMBGOXUU6LQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760364751 (33h7m14.456213514s) [noise=true , address=145.79.227.23:4546 , piece_id=73GYIV6FEN4GTBGNIXEO6NHFVBLWFEAKDW5MSHWM3PZVKUBEE63Q, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760496121 (33h7m14.456344874s) [noise=true , address=145.79.227.4:4185 , piece_id=PHN2D46K6N7EXQSFLLC653IXIPQ5KQURPGRKB5VIO2HQAGAHYS3A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760759412 (33h7m14.456608165s) [noise=true , address=145.79.227.8:4331 , piece_id=QD7MLX5QFRR7J5Z4AIOPFHU77SWSVWHW676F3IBPAU2BBGF6CQTQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760992276 (33h7m14.456841019s) [noise=true , address=145.79.227.24:4581 , piece_id=LX2A6M3I5YY4U2YDLFIFJ7RJA3QSFNHKWTW2SSRFQWKMVMSV55CA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.761119158 (33h7m14.456967911s) [noise=true , address=145.79.227.32:4835 , piece_id=YCG73FQ45UKT6XSC3YSN6R6CK2N25C34OV4LQ3PDGSKY7D5M334A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.761488600 (33h7m14.457337353s) [noise=true , address=145.79.227.17:4971 , piece_id=5VDD2RS3IHKDCMZYGEPCHMMAZMJ7G5LYIXLRQTC4KBKJDQ2MIH3Q, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.761536843 (33h7m14.457385586s) [noise=true , address=145.79.227.21:4474 , piece_id=APCGJFDNPWM4BJEEHP7EWQHQPRIHGW4NO7TMCGU32ELTRBF22LGA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.762222498 (33h7m14.458071241s) [noise=true , address=145.79.227.9:4366 , piece_id=I7GGALAWWWLA5HXX2EOPJCTIVZXI3YGLNE2K2R4ISPLU5MVXJNLA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:44.473460063 (33h7m16.169308816s) [upload=29065]: Next returned nil segment Oct 31 20:47:44.473587325 (33h7m16.169436088s) [upload=29065]: Waiting for error group managing segments... Oct 31 20:47:44.757605346 (33h7m16.453454089s) [noise=true , address=145.79.227.9:4366 , piece_id=I7GGALAWWWLA5HXX2EOPJCTIVZXI3YGLNE2K2R4ISPLU5MVXJNLA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.764557314 (33h7m16.460406067s) [noise=true , address=145.79.227.32:4835 , piece_id=YCG73FQ45UKT6XSC3YSN6R6CK2N25C34OV4LQ3PDGSKY7D5M334A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.774899719 (33h7m16.470748462s) [noise=true , address=145.79.227.4:4185 , piece_id=PHN2D46K6N7EXQSFLLC653IXIPQ5KQURPGRKB5VIO2HQAGAHYS3A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.775470495 (33h7m16.471319238s) [noise=true , address=145.79.227.10:4403 , piece_id=RIMPQPW53W2ZFWPUC2C6PHEAO4KYXOHEXYVP4OC5XSLKWARUUG5A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.794178068 (33h7m16.490026821s) [noise=true , address=145.79.227.24:4581 , piece_id=LX2A6M3I5YY4U2YDLFIFJ7RJA3QSFNHKWTW2SSRFQWKMVMSV55CA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.797767073 (33h7m16.493615816s) [noise=true , address=145.79.227.23:4546 , piece_id=73GYIV6FEN4GTBGNIXEO6NHFVBLWFEAKDW5MSHWM3PZVKUBEE63Q, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.809232004 (33h7m16.505080757s) [noise=true , address=145.79.227.8:4331 , piece_id=QD7MLX5QFRR7J5Z4AIOPFHU77SWSVWHW676F3IBPAU2BBGF6CQTQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.819266102 (33h7m16.515114855s) [noise=true , address=145.79.227.20:4438 , piece_id=KULG2SULBLTY3OFV5JGPQPPB647MA6G7GR543W4ITIWDTVWS5RNA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.828970591 (33h7m16.524819344s) [noise=true , address=145.79.227.13:4802 , piece_id=EX5NKH5RHPEXCEF2S3KYZ6DRMIL4AF5IIC6EQHLOVIUNVVCM3FWA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.841564723 (33h7m16.537413476s) [noise=true , address=145.79.227.17:4971 , piece_id=5VDD2RS3IHKDCMZYGEPCHMMAZMJ7G5LYIXLRQTC4KBKJDQ2MIH3Q, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.862610308 (33h7m16.558459051s) [noise=true , address=145.79.227.21:4474 , piece_id=APCGJFDNPWM4BJEEHP7EWQHQPRIHGW4NO7TMCGU32ELTRBF22LGA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:48:10.667054932 (33h7m42.362903695s) [noise=true , address=77.90.31.14:4661 , piece_id=FWKOERG4NXKSOIWJVFEJNV5XXOKRVTHM7DZFPDAYRK3T6WKFENHQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:48:11.815498164 (33h7m43.511346907s) [noise=true , address=45.91.116.8:4045 , piece_id=RBLYSRSYCLW4BM42HMCPWPUTIPZJTPOUBOFST2ZGTDRYRCMYPG4Q, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 188.166.225.170:49100->45.91.116.8:4045: use of closed network connection; write tcp 188.166.225.170:49100->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49100->45.91.116.8:4045: use of closed network connection Oct 31 20:48:11.815924096 (33h7m43.511772849s) [noise=true , address=77.90.31.17:4772 , piece_id=ZTNTL3LMNN6OTUSENUCIJ6QDPD7VDELTXJ634YAW5VJC6CE4EUDQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42670->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:48:11.816087947 (33h7m43.5119367s) [noise=true , address=77.90.31.16:4734 , piece_id=AJNH3MWGRYCYRVSYNKAR5OUWVP3QAFHCR4A4VUXHGIUQEYGLL52A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 188.166.225.170:55794->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:48:11.816286556 (33h7m43.512135309s) [noise=true , address=77.90.31.17:4770 , piece_id=ROS5JGE3AKSEPE74IA2VLZKLZ4SUZUVNNRS5G63RDTMBGOXUU6LQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 188.166.225.170:55648->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:48:11.816319889 (33h7m43.512168642s) [seg_pos={0 0}, upload=29065]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 20:48:12.061999186 (33h7m43.757847949s) [upload=29065]: Done uploading segments... Oct 31 20:48:12.062045554 (33h7m43.757894317s) [upload=29065]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.055853334s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.036984915s trace: https://jaeger.dev.storj.io/trace/5bed7337abd66ce6 FAIL (2025-10-31 20:48:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.037031133s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.249438407s trace: https://jaeger.dev.storj.io/trace/57ef7f61138e7c35 FAIL (2025-10-31 20:48:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 20:47:42.508656855 (33h7m14.204505608s) [upload=29065]: Starting upload Oct 31 20:47:42.508711358 (33h7m14.204560111s) [upload=29065]: Done starting upload Oct 31 20:47:42.508835555 (33h7m14.204684298s) [upload=29065]: Uploading segments... Oct 31 20:47:42.508919635 (33h7m14.204768378s) [upload=29065]: Got next segment. Inline: false Oct 31 20:47:42.508957166 (33h7m14.204805919s) [upload=29065]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 20:47:42.508981583 (33h7m14.204830336s) [upload=29065]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 20:47:42.757470730 (33h7m14.453319493s) [seg_pos={0 0}, upload=29065]: Begin upload segment... Oct 31 20:47:42.757527728 (33h7m14.453376471s) [seg_pos={0 0}, upload=29065]: Done begin upload segment. Oct 31 20:47:42.757564037 (33h7m14.45341278s) [noise=true , address=77.90.31.17:4772 , piece_id=ZTNTL3LMNN6OTUSENUCIJ6QDPD7VDELTXJ634YAW5VJC6CE4EUDQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.758551809 (33h7m14.454400572s) [noise=true , address=45.91.116.8:4045 , piece_id=RBLYSRSYCLW4BM42HMCPWPUTIPZJTPOUBOFST2ZGTDRYRCMYPG4Q, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.758883590 (33h7m14.454732353s) [noise=true , address=145.79.227.13:4802 , piece_id=EX5NKH5RHPEXCEF2S3KYZ6DRMIL4AF5IIC6EQHLOVIUNVVCM3FWA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759297398 (33h7m14.455146151s) [noise=true , address=145.79.227.10:4403 , piece_id=RIMPQPW53W2ZFWPUC2C6PHEAO4KYXOHEXYVP4OC5XSLKWARUUG5A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759395595 (33h7m14.455244338s) [noise=true , address=77.90.31.14:4661 , piece_id=FWKOERG4NXKSOIWJVFEJNV5XXOKRVTHM7DZFPDAYRK3T6WKFENHQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759665890 (33h7m14.455514643s) [noise=true , address=145.79.227.20:4438 , piece_id=KULG2SULBLTY3OFV5JGPQPPB647MA6G7GR543W4ITIWDTVWS5RNA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.759925394 (33h7m14.455774147s) [noise=true , address=77.90.31.16:4734 , piece_id=AJNH3MWGRYCYRVSYNKAR5OUWVP3QAFHCR4A4VUXHGIUQEYGLL52A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760006399 (33h7m14.455855142s) [noise=true , address=77.90.31.17:4770 , piece_id=ROS5JGE3AKSEPE74IA2VLZKLZ4SUZUVNNRS5G63RDTMBGOXUU6LQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760364751 (33h7m14.456213514s) [noise=true , address=145.79.227.23:4546 , piece_id=73GYIV6FEN4GTBGNIXEO6NHFVBLWFEAKDW5MSHWM3PZVKUBEE63Q, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760496121 (33h7m14.456344874s) [noise=true , address=145.79.227.4:4185 , piece_id=PHN2D46K6N7EXQSFLLC653IXIPQ5KQURPGRKB5VIO2HQAGAHYS3A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760759412 (33h7m14.456608165s) [noise=true , address=145.79.227.8:4331 , piece_id=QD7MLX5QFRR7J5Z4AIOPFHU77SWSVWHW676F3IBPAU2BBGF6CQTQ, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.760992276 (33h7m14.456841019s) [noise=true , address=145.79.227.24:4581 , piece_id=LX2A6M3I5YY4U2YDLFIFJ7RJA3QSFNHKWTW2SSRFQWKMVMSV55CA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.761119158 (33h7m14.456967911s) [noise=true , address=145.79.227.32:4835 , piece_id=YCG73FQ45UKT6XSC3YSN6R6CK2N25C34OV4LQ3PDGSKY7D5M334A, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.761488600 (33h7m14.457337353s) [noise=true , address=145.79.227.17:4971 , piece_id=5VDD2RS3IHKDCMZYGEPCHMMAZMJ7G5LYIXLRQTC4KBKJDQ2MIH3Q, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.761536843 (33h7m14.457385586s) [noise=true , address=145.79.227.21:4474 , piece_id=APCGJFDNPWM4BJEEHP7EWQHQPRIHGW4NO7TMCGU32ELTRBF22LGA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:42.762222498 (33h7m14.458071241s) [noise=true , address=145.79.227.9:4366 , piece_id=I7GGALAWWWLA5HXX2EOPJCTIVZXI3YGLNE2K2R4ISPLU5MVXJNLA, seg_pos={0 0}, upload=29065]: Uploading piece... Oct 31 20:47:44.473460063 (33h7m16.169308816s) [upload=29065]: Next returned nil segment Oct 31 20:47:44.473587325 (33h7m16.169436088s) [upload=29065]: Waiting for error group managing segments... Oct 31 20:47:44.757605346 (33h7m16.453454089s) [noise=true , address=145.79.227.9:4366 , piece_id=I7GGALAWWWLA5HXX2EOPJCTIVZXI3YGLNE2K2R4ISPLU5MVXJNLA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.764557314 (33h7m16.460406067s) [noise=true , address=145.79.227.32:4835 , piece_id=YCG73FQ45UKT6XSC3YSN6R6CK2N25C34OV4LQ3PDGSKY7D5M334A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.774899719 (33h7m16.470748462s) [noise=true , address=145.79.227.4:4185 , piece_id=PHN2D46K6N7EXQSFLLC653IXIPQ5KQURPGRKB5VIO2HQAGAHYS3A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.775470495 (33h7m16.471319238s) [noise=true , address=145.79.227.10:4403 , piece_id=RIMPQPW53W2ZFWPUC2C6PHEAO4KYXOHEXYVP4OC5XSLKWARUUG5A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.794178068 (33h7m16.490026821s) [noise=true , address=145.79.227.24:4581 , piece_id=LX2A6M3I5YY4U2YDLFIFJ7RJA3QSFNHKWTW2SSRFQWKMVMSV55CA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.797767073 (33h7m16.493615816s) [noise=true , address=145.79.227.23:4546 , piece_id=73GYIV6FEN4GTBGNIXEO6NHFVBLWFEAKDW5MSHWM3PZVKUBEE63Q, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.809232004 (33h7m16.505080757s) [noise=true , address=145.79.227.8:4331 , piece_id=QD7MLX5QFRR7J5Z4AIOPFHU77SWSVWHW676F3IBPAU2BBGF6CQTQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.819266102 (33h7m16.515114855s) [noise=true , address=145.79.227.20:4438 , piece_id=KULG2SULBLTY3OFV5JGPQPPB647MA6G7GR543W4ITIWDTVWS5RNA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.828970591 (33h7m16.524819344s) [noise=true , address=145.79.227.13:4802 , piece_id=EX5NKH5RHPEXCEF2S3KYZ6DRMIL4AF5IIC6EQHLOVIUNVVCM3FWA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.841564723 (33h7m16.537413476s) [noise=true , address=145.79.227.17:4971 , piece_id=5VDD2RS3IHKDCMZYGEPCHMMAZMJ7G5LYIXLRQTC4KBKJDQ2MIH3Q, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:47:44.862610308 (33h7m16.558459051s) [noise=true , address=145.79.227.21:4474 , piece_id=APCGJFDNPWM4BJEEHP7EWQHQPRIHGW4NO7TMCGU32ELTRBF22LGA, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:48:10.667054932 (33h7m42.362903695s) [noise=true , address=77.90.31.14:4661 , piece_id=FWKOERG4NXKSOIWJVFEJNV5XXOKRVTHM7DZFPDAYRK3T6WKFENHQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: <nil> Oct 31 20:48:11.815498164 (33h7m43.511346907s) [noise=true , address=45.91.116.8:4045 , piece_id=RBLYSRSYCLW4BM42HMCPWPUTIPZJTPOUBOFST2ZGTDRYRCMYPG4Q, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 188.166.225.170:49100->45.91.116.8:4045: use of closed network connection; write tcp 188.166.225.170:49100->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49100->45.91.116.8:4045: use of closed network connection Oct 31 20:48:11.815924096 (33h7m43.511772849s) [noise=true , address=77.90.31.17:4772 , piece_id=ZTNTL3LMNN6OTUSENUCIJ6QDPD7VDELTXJ634YAW5VJC6CE4EUDQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: write tcp 188.166.225.170:42670->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:48:11.816087947 (33h7m43.5119367s) [noise=true , address=77.90.31.16:4734 , piece_id=AJNH3MWGRYCYRVSYNKAR5OUWVP3QAFHCR4A4VUXHGIUQEYGLL52A, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 188.166.225.170:55794->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:48:11.816286556 (33h7m43.512135309s) [noise=true , address=77.90.31.17:4770 , piece_id=ROS5JGE3AKSEPE74IA2VLZKLZ4SUZUVNNRS5G63RDTMBGOXUU6LQ, seg_pos={0 0}, upload=29065]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 188.166.225.170:55648->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:48:11.816319889 (33h7m43.512168642s) [seg_pos={0 0}, upload=29065]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 20:48:12.061999186 (33h7m43.757847949s) [upload=29065]: Done uploading segments... Oct 31 20:48:12.062045554 (33h7m43.757894317s) [upload=29065]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.249454698s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.0023855s trace: https://jaeger.dev.storj.io/trace/5bed7337abd66ce6 FAIL (2025-10-31 20:48:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.002409046s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.250325983s trace: https://jaeger.dev.storj.io/trace/728a8f617fe1193 FAIL (2025-10-31 20:47:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 31 20:47:12.033717731 (33h6m43.729566474s) [upload=29058]: Starting upload Oct 31 20:47:12.033879910 (33h6m43.729728663s) [upload=29058]: Done starting upload Oct 31 20:47:12.033986082 (33h6m43.729834835s) [upload=29058]: Uploading segments... Oct 31 20:47:12.034114055 (33h6m43.729962808s) [upload=29058]: Got next segment. Inline: false Oct 31 20:47:12.034150184 (33h6m43.729998927s) [upload=29058]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 31 20:47:12.034155796 (33h6m43.730004549s) [upload=29058]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 31 20:47:12.283488487 (33h6m43.979337241s) [seg_pos={0 0}, upload=29058]: Begin upload segment... Oct 31 20:47:12.283540517 (33h6m43.97938926s) [seg_pos={0 0}, upload=29058]: Done begin upload segment. Oct 31 20:47:12.283569152 (33h6m43.979417905s) [noise=true , address=145.79.227.21:4473 , piece_id=DPQSTZYSO4F5X6BCUH5JGUYQAO3XAFKDA6PXEMAQUA3RCRT26RWA, seg_pos={0 0}, upload=29058]: Uploading piece... Oct 31 20:47:12.283639395 (33h6m43.979488149s) [noise=true , address=77.90.31.12:4589 , piece_id=PJUOWGWAXR6KVOFZKASEPGGTYKYZBSUCRFMUXA52APKJHD6QXTJQ, seg_pos={0 0}, upload=29058]: Uploading piece... Oct 31 20:47:12.284176107 (33h6m43.98002486s) [noise=true , address=145.79.227.28:4726 , piece_id=S4E5FOMNPZBUVLZBQECJ3QFWIOU7734ZHTTHYTDFT3NT3A3YVAMQ, seg_pos={0 0}, upload=29058]: Uploading piece... Oct 31 20:47:12.284671701 (33h6m43.980520444s) [noise=true , address=145.79.227.23:4546 , piece_id=3KJWZ76XQJGUWKI5KWCCH7TQTKVR4YE2OJDRQDKSTEMECB342IIA, seg_pos={0 0}, upload=29058]: Uploading piece... Oct 31 20:47:12.285032228 (33h6m43.980880981s) [noise=true , address=145.79.227.8:4331 , piece_id=EBLAKTE6KKGCCPA37UWBIGDTYJR5FCUFFHLY2UKL4XBSGNH4Z6IQ, seg_pos={0 0}, upload=29058]: Uploading piece... Oct 31 20:47:12.285381332 (33h6m43.981230085s) [noise=true , address=77.90.31.16:4734 , piece_id=6CCDDXXCSK7LNMF6YCKMYQDPNY5M6GGAHZT4QEPRNSHKHZ47N3YA, seg_pos={0 0}, upload=29058]: Uploading piece... Oct 31 20:47:12.285743291 (33h6m43.981592034s) [noise=true , address=145.79.227.16:4912 , piece_id=P2LS7GMTVVAIWRXM4BHS77U55RVWJ2B2A77NUMRFFPYBOFJ4ZP6A, seg_pos={0 0}, upload=29058]: Uploading piece... Oct 31 20:47:12.286141500 (33h6m43.981990243s) [noise=true , address=145.79.227.20:4437 , piece_id=B6HHYY6VYCT7WHR5VWSQ54VW5R2OOFLG5Y3DBR6676ZG766ABDEA, seg_pos={0 0}, upload=29058]: Uploading piece... Oct 31 20:47:12.286520071 (33h6m43.982368824s) [noise=true , address=45.91.116.8:4045 , piece_id=DQQQAU4NLJ2DD32AVQCJJ22V66DDFCQXXNPS6DHFY52SIJGQZ7BA, seg_pos={0 0}, upload=29058]: Uploading piece... Oct 31 20:47:12.287058235 (33h6m43.982906978s) [noise=true , address=145.79.227.32:4834 , piece_id=EHXF7BGIKUD5SJATKTED5X6NPWTG6RRACPON4R4F2MCE6DU66YLA, seg_pos={0 0}, upload=29058]: Uploading piece... Oct 31 20:47:12.287469949 (33h6m43.983318693s) [noise=true , address=145.79.227.10:4402 , piece_id=ES46SJCLVEJ3A3CIXJ5UES6EW2NVTIVHA5RKZ4VJEC4XGWMR6EZA, seg_pos={0 0}, upload=29058]: Uploading piece... Oct 31 20:47:12.287894197 (33h6m43.98374295s) [noise=true , address=145.79.227.11:4729 , piece_id=T2MMV2QKT2TIH77SEVFALKYHR2RZ3RZYYMEJSMYIM56GWHPI4TJQ, seg_pos={0 0}, upload=29058]: Uploading piece... Oct 31 20:47:12.288349724 (33h6m43.984198478s) [noise=true , address=145.79.227.15:4875 , piece_id=AHXIUPVHPYHSJFM45IFGJAVRJOPFZCJDKRF74F37HO7G7U3FWMIQ, seg_pos={0 0}, upload=29058]: Uploading piece... Oct 31 20:47:12.288657461 (33h6m43.984506214s) [noise=true , address=145.79.227.17:4969 , piece_id=CY6CKENFKSXXLJDE2FHK6QM7IDTJFFJKXU3CL4F4B4E7M32W44DQ, seg_pos={0 0}, upload=29058]: Uploading piece... Oct 31 20:47:12.289305735 (33h6m43.985154539s) [noise=true , address=145.79.227.9:4365 , piece_id=VXITVTEMAHVH5A5TAHVMQBZ3ICCTPESV52ZRTJA74MWKFCPRHJKA, seg_pos={0 0}, upload=29058]: Uploading piece... Oct 31 20:47:12.289937418 (33h6m43.985786171s) [noise=true , address=77.90.31.11:4554 , piece_id=FGVJ6KAPBO5HUEPMG2ZYARVOSE5FBWTXQUTMWI7VAFKCBTTUJB6A, seg_pos={0 0}, upload=29058]: Uploading piece... Oct 31 20:47:13.998966217 (33h6m45.69481497s) [upload=29058]: Next returned nil segment Oct 31 20:47:13.998989801 (33h6m45.694838554s) [upload=29058]: Waiting for error group managing segments... Oct 31 20:47:14.284106384 (33h6m45.979955127s) [noise=true , address=145.79.227.9:4365 , piece_id=VXITVTEMAHVH5A5TAHVMQBZ3ICCTPESV52ZRTJA74MWKFCPRHJKA, seg_pos={0 0}, upload=29058]: Done uploading piece. err: <nil> Oct 31 20:47:14.298504150 (33h6m45.994352893s) [noise=true , address=145.79.227.15:4875 , piece_id=AHXIUPVHPYHSJFM45IFGJAVRJOPFZCJDKRF74F37HO7G7U3FWMIQ, seg_pos={0 0}, upload=29058]: Done uploading piece. err: <nil> Oct 31 20:47:14.299365340 (33h6m45.995214083s) [noise=true , address=145.79.227.20:4437 , piece_id=B6HHYY6VYCT7WHR5VWSQ54VW5R2OOFLG5Y3DBR6676ZG766ABDEA, seg_pos={0 0}, upload=29058]: Done uploading piece. err: <nil> Oct 31 20:47:14.304079668 (33h6m45.999928421s) [noise=true , address=145.79.227.16:4912 , piece_id=P2LS7GMTVVAIWRXM4BHS77U55RVWJ2B2A77NUMRFFPYBOFJ4ZP6A, seg_pos={0 0}, upload=29058]: Done uploading piece. err: <nil> Oct 31 20:47:14.304835647 (33h6m46.000684391s) [noise=true , address=145.79.227.8:4331 , piece_id=EBLAKTE6KKGCCPA37UWBIGDTYJR5FCUFFHLY2UKL4XBSGNH4Z6IQ, seg_pos={0 0}, upload=29058]: Done uploading piece. err: <nil> Oct 31 20:47:14.308055027 (33h6m46.00390378s) [noise=true , address=145.79.227.17:4969 , piece_id=CY6CKENFKSXXLJDE2FHK6QM7IDTJFFJKXU3CL4F4B4E7M32W44DQ, seg_pos={0 0}, upload=29058]: Done uploading piece. err: <nil> Oct 31 20:47:14.310465077 (33h6m46.00631383s) [noise=true , address=145.79.227.11:4729 , piece_id=T2MMV2QKT2TIH77SEVFALKYHR2RZ3RZYYMEJSMYIM56GWHPI4TJQ, seg_pos={0 0}, upload=29058]: Done uploading piece. err: <nil> Oct 31 20:47:14.319456558 (33h6m46.015305311s) [noise=true , address=145.79.227.10:4402 , piece_id=ES46SJCLVEJ3A3CIXJ5UES6EW2NVTIVHA5RKZ4VJEC4XGWMR6EZA, seg_pos={0 0}, upload=29058]: Done uploading piece. err: <nil> Oct 31 20:47:14.324327934 (33h6m46.020176697s) [noise=true , address=145.79.227.23:4546 , piece_id=3KJWZ76XQJGUWKI5KWCCH7TQTKVR4YE2OJDRQDKSTEMECB342IIA, seg_pos={0 0}, upload=29058]: Done uploading piece. err: <nil> Oct 31 20:47:14.334230421 (33h6m46.030079174s) [noise=true , address=145.79.227.32:4834 , piece_id=EHXF7BGIKUD5SJATKTED5X6NPWTG6RRACPON4R4F2MCE6DU66YLA, seg_pos={0 0}, upload=29058]: Done uploading piece. err: <nil> Oct 31 20:47:14.351242716 (33h6m46.047091469s) [noise=true , address=145.79.227.21:4473 , piece_id=DPQSTZYSO4F5X6BCUH5JGUYQAO3XAFKDA6PXEMAQUA3RCRT26RWA, seg_pos={0 0}, upload=29058]: Done uploading piece. err: <nil> Oct 31 20:47:14.390142760 (33h6m46.085991503s) [noise=true , address=145.79.227.28:4726 , piece_id=S4E5FOMNPZBUVLZBQECJ3QFWIOU7734ZHTTHYTDFT3NT3A3YVAMQ, seg_pos={0 0}, upload=29058]: Done uploading piece. err: <nil> Oct 31 20:47:41.340514050 (33h7m13.036362803s) [noise=true , address=45.91.116.8:4045 , piece_id=DQQQAU4NLJ2DD32AVQCJJ22V66DDFCQXXNPS6DHFY52SIJGQZ7BA, seg_pos={0 0}, upload=29058]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 188.166.225.170:58564->45.91.116.8:4045: use of closed network connection; write tcp 188.166.225.170:58564->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:58564->45.91.116.8:4045: use of closed network connection Oct 31 20:47:41.340626594 (33h7m13.036475347s) [noise=true , address=77.90.31.11:4554 , piece_id=FGVJ6KAPBO5HUEPMG2ZYARVOSE5FBWTXQUTMWI7VAFKCBTTUJB6A, seg_pos={0 0}, upload=29058]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: write tcp 188.166.225.170:46416->77.90.31.11:4554: use of closed network connection; write tcp 188.166.225.170:46416->77.90.31.11:4554: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46416->77.90.31.11:4554: use of closed network connection Oct 31 20:47:41.340659196 (33h7m13.036507939s) [noise=true , address=77.90.31.16:4734 , piece_id=6CCDDXXCSK7LNMF6YCKMYQDPNY5M6GGAHZT4QEPRNSHKHZ47N3YA, seg_pos={0 0}, upload=29058]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: write tcp 188.166.225.170:49572->77.90.31.16:4734: use of closed network connection; write tcp 188.166.225.170:49572->77.90.31.16:4734: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49572->77.90.31.16:4734: use of closed network connection Oct 31 20:47:41.341122708 (33h7m13.036971461s) [noise=true , address=77.90.31.12:4589 , piece_id=PJUOWGWAXR6KVOFZKASEPGGTYKYZBSUCRFMUXA52APKJHD6QXTJQ, seg_pos={0 0}, upload=29058]: Done uploading piece. err: ecclient: upload failed (node:1jHm28wBr3S4P5omoHpbVbyYpnXvxakxkkmgVm5fthtj4HZbv, address:77.90.31.12:4589): protocol: read tcp 188.166.225.170:41936->77.90.31.12:4589: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 31 20:47:41.341150992 (33h7m13.036999735s) [seg_pos={0 0}, upload=29058]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 31 20:47:41.587061018 (33h7m13.282909771s) [upload=29058]: Done uploading segments... Oct 31 20:47:41.587101766 (33h7m13.282950519s) [upload=29058]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.250349478s core uplink us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 610.235184ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/5f51b4294ae22a4d FAIL (2025-10-31 19:52:42 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:36538->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 610.26467ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 297.07614ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/5f51b4294ae22a4d FAIL (2025-10-31 19:51:55 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:36538->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 297.115183ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 319.944527ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/5f51b4294ae22a4d FAIL (2025-10-31 19:51:40 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:36538->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 319.970265ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 341.501425ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/5f51b4294ae22a4d FAIL (2025-10-31 19:51:12 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:36538->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 341.529498ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 308.04158ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/5f51b4294ae22a4d FAIL (2025-10-31 19:50:08 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:36538->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 308.068942ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 796.967971ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/5f51b4294ae22a4d FAIL (2025-10-31 19:49:17 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:36538->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 796.984051ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 319.936372ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/5f51b4294ae22a4d FAIL (2025-10-31 19:48:55 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:36538->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 319.969224ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 329.25735ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/5f51b4294ae22a4d FAIL (2025-10-31 19:48:22 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:36538->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 329.281615ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 342.063595ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/5f51b4294ae22a4d FAIL (2025-10-31 19:47:52 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:36538->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 342.096426ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 607.647997ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/5f51b4294ae22a4d FAIL (2025-10-31 19:46:40 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:36538->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 607.678655ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 322.645914ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/5f51b4294ae22a4d FAIL (2025-10-31 19:46:20 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:36538->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 322.678214ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 328.978693ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/5f51b4294ae22a4d FAIL (2025-10-31 19:45:24 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:36538->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 329.007948ms core website

Orbiter Check History

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)