Orbiter History

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 309.670095ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/67db01bf0ac8b2f2 FAIL (2025-11-03 00:24:06 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:58958->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 309.69425ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 373.139122ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/67db01bf0ac8b2f2 FAIL (2025-11-03 00:23:35 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:58958->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 373.17488ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 335.643723ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/67db01bf0ac8b2f2 FAIL (2025-11-03 00:22:59 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:58958->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 335.67368ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 335.797337ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/67db01bf0ac8b2f2 FAIL (2025-11-03 00:22:02 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:58958->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 335.827705ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 49.061884ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/75343aeac90325c8 FAIL (2025-11-03 00:21:57 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:56576->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 49.089265ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 42.656397ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/75343aeac90325c8 FAIL (2025-11-03 00:21:37 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:56576->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 42.682396ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 54.61619ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/75343aeac90325c8 FAIL (2025-11-03 00:21:06 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:56576->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 54.64225ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 315.661231ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/67db01bf0ac8b2f2 FAIL (2025-11-03 00:20:54 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:58958->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 315.694924ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 47.19575ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/75343aeac90325c8 FAIL (2025-11-03 00:20:29 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:56576->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 47.224464ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.............X.] duration: 320.45029ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/67db01bf0ac8b2f2 FAIL (2025-11-03 00:20:02 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:58958->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 320.513199ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 47.625468ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/75343aeac90325c8 FAIL (2025-11-03 00:19:46 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:56576->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 47.650195ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..............X] duration: 281.158051ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/67db01bf0ac8b2f2 FAIL (2025-11-03 00:19:23 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:58958->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 281.17819ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 64.339085ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/75343aeac90325c8 FAIL (2025-11-03 00:19:17 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:56576->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 64.366778ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 370.076659ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/75343aeac90325c8 FAIL (2025-11-03 00:18:46 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:56576->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 370.121243ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 60.183491ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/75343aeac90325c8 FAIL (2025-11-03 00:18:02 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:56576->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 60.208559ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 58.881492ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/75343aeac90325c8 FAIL (2025-11-03 00:17:29 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:56576->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 58.90674ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 49.186981ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/75343aeac90325c8 FAIL (2025-11-03 00:17:04 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:56576->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 49.218221ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 610.748115ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/428e4b0dbce438f0 FAIL (2025-11-03 00:17:03 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:59614->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 610.792099ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.731222822s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-03 00:16:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.731248451s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 53.809464ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/75343aeac90325c8 FAIL (2025-11-03 00:16:35 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:56576->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 53.826215ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 335.846887ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/428e4b0dbce438f0 FAIL (2025-11-03 00:16:14 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:59614->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 335.877277ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 258.201135ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/75343aeac90325c8 FAIL (2025-11-03 00:15:58 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:56576->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 258.224509ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 349.472443ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/428e4b0dbce438f0 FAIL (2025-11-03 00:15:54 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:59614->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 349.593645ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.623250918s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-03 00:15:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.623263942s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 46.495355ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/75343aeac90325c8 FAIL (2025-11-03 00:15:15 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:56576->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 46.527216ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...X...........] duration: 329.791467ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/428e4b0dbce438f0 FAIL (2025-11-03 00:15:14 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:59614->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 329.832956ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.723090922s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-03 00:14:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.723124786s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.............X.] duration: 59.221751ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/75343aeac90325c8 FAIL (2025-11-03 00:14:28 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:56576->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 59.248241ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [....X..........] duration: 307.818207ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/428e4b0dbce438f0 FAIL (2025-11-03 00:14:08 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:59614->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 307.835469ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [..............X] duration: 227.410787ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/75343aeac90325c8 FAIL (2025-11-03 00:14:04 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:56576->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 227.439281ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.760487883s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-03 00:13:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.76051267s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.....X.........] duration: 310.987672ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/428e4b0dbce438f0 FAIL (2025-11-03 00:13:31 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:59614->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 311.0133ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [......X........] duration: 307.381809ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/428e4b0dbce438f0 FAIL (2025-11-03 00:12:48 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:59614->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 307.410873ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.525023643s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-03 00:12:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.52504805s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.......X.......] duration: 628.656136ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/428e4b0dbce438f0 FAIL (2025-11-03 00:12:05 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:59614->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 628.679009ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.743209196s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-03 00:11:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.743236017s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [........X......] duration: 317.929122ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/428e4b0dbce438f0 FAIL (2025-11-03 00:11:33 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:59614->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 317.960921ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.698231465s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-03 00:10:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.698260912s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.........X.....] duration: 574.0633ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/428e4b0dbce438f0 FAIL (2025-11-03 00:10:34 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:59614->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 574.09037ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.692195984s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-03 00:10:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.692232494s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.642895115s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-03 00:09:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.642926104s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 317.323387ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/428e4b0dbce438f0 FAIL (2025-11-03 00:09:25 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:59614->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 317.347553ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 315.892951ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/428e4b0dbce438f0 FAIL (2025-11-03 00:08:53 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:59614->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 315.921587ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.710444712s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-03 00:08:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.710471294s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 317.725582ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/428e4b0dbce438f0 FAIL (2025-11-03 00:08:17 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:59614->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 317.737013ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.718237911s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-03 00:08:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.718290271s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.............X.] duration: 330.924365ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/428e4b0dbce438f0 FAIL (2025-11-03 00:07:24 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:59614->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 330.957678ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.747458662s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-03 00:06:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.747486755s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.715582484s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-03 00:06:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.715610006s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..............X] duration: 231.307004ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/428e4b0dbce438f0 FAIL (2025-11-03 00:06:14 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:59614->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 231.334465ms core website

Orbiter Check History

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)