Orbiter History

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [X....X.........] duration: 43.370382ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/61211775172e1034 FAIL (2025-11-02 23:42:43 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:57284->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 43.398144ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.82401158s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-02 23:42:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.824034835s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [.X....X........] duration: 58.436281ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/61211775172e1034 FAIL (2025-11-02 23:42:01 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:57284->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 58.470135ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.622627966s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-02 23:41:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.622659026s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [..X....X.......] duration: 68.92784ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/61211775172e1034 FAIL (2025-11-02 23:41:13 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:57284->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 68.958627ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.619272311s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-02 23:40:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.619303279s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [...X....X......] duration: 63.836953ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/61211775172e1034 FAIL (2025-11-02 23:40:06 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:57284->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 63.858436ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.6635003s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-02 23:39:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.66352603s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.772186725s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-02 23:39:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.772220028s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [....X....X.....] duration: 64.843942ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/61211775172e1034 FAIL (2025-11-02 23:39:03 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:57284->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 64.889127ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.954165694s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-02 23:38:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.954215088s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [.....X....X....] duration: 70.127252ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/61211775172e1034 FAIL (2025-11-02 23:37:51 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:57284->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 70.153584ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [......X....X...] duration: 62.465422ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/61211775172e1034 FAIL (2025-11-02 23:36:55 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:57284->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 62.586124ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.754131311s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-02 23:36:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.754159335s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.639226745s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-02 23:35:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.639336837s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [.......X....X..] duration: 58.181406ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/61211775172e1034 FAIL (2025-11-02 23:35:54 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:57284->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 58.211412ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [........X....X.] duration: 61.99642ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/61211775172e1034 FAIL (2025-11-02 23:35:35 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:57284->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 62.020887ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.767253265s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-02 23:35:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.76728254s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 2 out of 15 is failed (threshold=6) [.........X....X] duration: 218.633701ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/61211775172e1034 FAIL (2025-11-02 23:34:50 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:57284->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 218.672625ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.695694178s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-02 23:34:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.695833663s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [..........X....] duration: 48.285622ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/1021bcd3774151c2 FAIL (2025-11-02 23:33:46 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:35382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 48.313466ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.758766296s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-02 23:33:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.758793448s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [...........X...] duration: 49.611969ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/1021bcd3774151c2 FAIL (2025-11-02 23:32:41 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:35382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 49.640062ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.734991148s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-02 23:32:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.735017688s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 44.733579ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/1021bcd3774151c2 FAIL (2025-11-02 23:32:09 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:35382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 44.760641ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.675241977s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-02 23:32:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.675272585s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.579212923s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-02 23:31:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.579228853s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 2m23.145977206s trace: https://jaeger.dev.storj.io/trace/1c92b49415ecb928 FAIL (2025-11-02 23:31:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== VYA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.025568338 (83h48m49.721417091s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026574943 (83h48m49.722423696s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.026989102 (83h48m49.722837855s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027131293 (83h48m49.722980046s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027301487 (83h48m49.72315023s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027669318 (83h48m49.723518061s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.027925966 (83h48m49.723774719s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028168067 (83h48m49.72401682s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028467808 (83h48m49.724316561s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028536659 (83h48m49.724385403s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.028825659 (83h48m49.724674403s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029150588 (83h48m49.724999331s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029303059 (83h48m49.725151802s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.029846713 (83h48m49.725695466s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.030504356 (83h48m49.726353109s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.031289852 (83h48m49.727138605s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:18.032065909 (83h48m49.727914662s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Uploading piece... Nov 2 23:29:19.364465732 (83h48m51.060314475s) [upload=73800]: Next returned nil segment Nov 2 23:29:19.364484678 (83h48m51.060333431s) [upload=73800]: Waiting for error group managing segments... Nov 2 23:29:19.703170247 (83h48m51.39901899s) [noise=true , address=64.49.13.170:4728 , piece_id=G7J7OO2OJU3ANKWJXVY5YHLV6OOXOK6Q5GEFW5QZIQ6RE426UI4Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.913752007 (83h48m51.60960075s) [noise=true , address=45.91.116.4:4941 , piece_id=DAQO4NLZ6P6E5PC4K2CKFGTTNLMFLKGST4DDQ6JYXU2YOIDTH4UA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.920797773 (83h48m51.616646526s) [noise=true , address=145.79.227.20:4441 , piece_id=6Y4GA4LPVDLN2L776ZB3S3WHRJNO7KJXQWPPYEBHYTOL77L7MI4A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.926354834 (83h48m51.622203587s) [noise=true , address=45.91.116.13:4169 , piece_id=AVX2KU63XTQJZISFCTNMJDLE4B3VIORPO675A2IBTIBBRUTII47Q, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.936127022 (83h48m51.631975775s) [noise=true , address=145.79.227.2:4138 , piece_id=4HVZWAQQQKG2EZS4EOHH5H6ZZ2TTWOTUG6R7SVBB3HPX5TSY54KA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.940438282 (83h48m51.636287035s) [noise=true , address=145.79.227.3:4180 , piece_id=USNSZ4HD7XK2MZTEJD6Z3XCUMB4GPX4ZYW62QP2UBFDZZ2UGFZPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.950386424 (83h48m51.646235177s) [noise=true , address=45.91.116.7:4027 , piece_id=6E5U3YF6L7AEO7PX636QUYUJZF7UDQXITH3B4ESX6UDOFJ6X6VYA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.963359988 (83h48m51.659208741s) [noise=true , address=145.79.227.37:4590 , piece_id=6I6FLWWFPAIY24PENRJFKZRNQZTPEIZBD7GKD4CWMG245IKSOCJA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.974894952 (83h48m51.670743695s) [noise=true , address=145.79.227.30:4793 , piece_id=ZOEWS5X74C4OCMPP27DLOEY2C4SDKQTSGQSHP3DGIRUWIIYNA4CA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.981536419 (83h48m51.677385172s) [noise=true , address=145.79.227.46:4629 , piece_id=IVJGM5SF42WFQ3BOTH4LWMV5DXH2PNAV5HGFZWHSG4RUHIVXRYBQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.989595594 (83h48m51.685444337s) [noise=true , address=145.79.227.26:4659 , piece_id=WGULHTMRG7S6XPNHENQU6UWX4BBJ73SDJYKFSTIEUUVUQ4ECJFPA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:19.991162779 (83h48m51.687011522s) [noise=true , address=145.79.227.38:4010 , piece_id=ERVNGVII3R4M5IOOQL7RBHD4SBHYXG5FZUZFALARPD4A65GI267A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.002910960 (83h48m51.698759713s) [noise=true , address=145.79.227.7:4297 , piece_id=QUNXSM5CQQOFLOOHKOGWM56NYC7LAK2BQCSH3TVUA47OD7VZV4AQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003028834 (83h48m51.698877587s) [noise=true , address=145.79.227.19:4110 , piece_id=AIX4VNQOU27VNSZVURIMJF5KIJWZINUL4ANW2LWRFWCP5GR55VFA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: <nil> Nov 2 23:29:20.003050876 (83h48m51.698899619s) [seg_pos={0 0}, upload=73800]: Segment reached optimal threshold of 14 pieces. Nov 2 23:29:20.003272728 (83h48m51.699121471s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1LLSnWuMa6W7y2jhFUqXoSPEsTyrFsKi8FQDNXF8Tv7XHbP6yt): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003279461 (83h48m51.699128204s) [noise=true , address=77.90.31.13:4657 , piece_id=CG65J5K7OSRMEOEQMXK2LAYBNJSL5X2FBJFQCPHBUPXL3OZ6Q57A, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003340998 (83h48m51.699189761s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BdCdPHn6ko7nfY4sPuc4NutXABzJDdHHhtSnyXri5gofhZEZJ): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003356487 (83h48m51.69920524s) [noise=true , address=145.79.227.16:4943 , piece_id=QKQL54CFFEVESYISEFWII6JHJPWOVUFOJW7FQTSMNSDR7IQLGUAA, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003407134 (83h48m51.699255887s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003414167 (83h48m51.69926291s) [noise=true , address=45.91.116.10:4107 , piece_id=5GIM4CMSKFBF4ASXUMUSH6O2RW6LCATJYUFU64SNRMOWNTGUAQPQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003536790 (83h48m51.699385543s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Nov 2 23:29:20.003546540 (83h48m51.699395293s) [noise=true , address=45.91.116.6:4005 , piece_id=EGIVB5IDQZAL5BARZHIC33HYSIDX6OC4ONTDGFJAKZSSCLRWYNNQ, seg_pos={0 0}, upload=73800]: Long tail cancellation due to optimal threshold Nov 2 23:29:20.003607967 (83h48m51.69945671s) [seg_pos={0 0}, upload=73800]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 2 23:29:20.003810993 (83h48m51.699659746s) [upload=73800]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 2 23:29:20.003821754 (83h48m51.699670497s) [upload=73800]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 2 23:31:40.226202497 (83h51m11.92205124s) [upload=73800]: Done uploading segments... Nov 2 23:31:40.226271067 (83h51m11.92211981s) [upload=73800]: Upload finished. err: metaclient: context deadline exceeded Duration: 2m23.145996893s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.............X.] duration: 43.71996ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/1021bcd3774151c2 FAIL (2025-11-02 23:31:24 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:35382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 43.74576ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [..............X] duration: 424.052597ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/1021bcd3774151c2 FAIL (2025-11-02 23:30:51 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:35382->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 424.067226ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 310.245237ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3efcd5c7086a8af1 FAIL (2025-11-02 23:05:56 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42728->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 310.274362ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 311.030893ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3efcd5c7086a8af1 FAIL (2025-11-02 23:05:13 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42728->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 311.063354ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..X............] duration: 317.313979ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3efcd5c7086a8af1 FAIL (2025-11-02 23:04:31 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42728->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 317.350458ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [X..X...........] duration: 320.872202ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3efcd5c7086a8af1 FAIL (2025-11-02 23:03:56 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42728->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 320.898142ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [.X..X..........] duration: 306.823584ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3efcd5c7086a8af1 FAIL (2025-11-02 23:03:18 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42728->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 306.852641ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [..X..X.........] duration: 322.617086ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3efcd5c7086a8af1 FAIL (2025-11-02 23:02:39 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42728->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 322.644367ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [...X..X........] duration: 308.548416ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3efcd5c7086a8af1 FAIL (2025-11-02 23:01:46 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42728->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 308.575527ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [....X..X.......] duration: 322.99821ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3efcd5c7086a8af1 FAIL (2025-11-02 23:01:29 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42728->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 323.025112ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [.....X..X......] duration: 320.363574ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3efcd5c7086a8af1 FAIL (2025-11-02 23:00:28 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42728->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 320.390797ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [......X..X.....] duration: 321.82621ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3efcd5c7086a8af1 FAIL (2025-11-02 23:00:02 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42728->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 321.864573ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [.......X..X....] duration: 306.935074ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3efcd5c7086a8af1 FAIL (2025-11-02 22:59:11 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42728->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 306.963146ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [........X..X...] duration: 319.215148ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3efcd5c7086a8af1 FAIL (2025-11-02 22:58:29 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42728->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 319.242479ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [.........X..X..] duration: 315.343894ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3efcd5c7086a8af1 FAIL (2025-11-02 22:58:02 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42728->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 315.374383ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [..........X..X.] duration: 320.695146ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3efcd5c7086a8af1 FAIL (2025-11-02 22:56:49 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42728->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 320.722068ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 2 out of 15 is failed (threshold=6) [...........X..X] duration: 239.822569ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/3efcd5c7086a8af1 FAIL (2025-11-02 22:56:13 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42728->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 239.834481ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [............X..] duration: 334.239045ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/7f01564231bd55f9 FAIL (2025-11-02 22:55:42 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42724->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 334.264414ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [.............X.] duration: 308.801789ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/7f01564231bd55f9 FAIL (2025-11-02 22:54:43 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42724->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 308.826616ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [..............X] duration: 238.482518ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/7f01564231bd55f9 FAIL (2025-11-02 22:54:26 UTC) orbiter warning: Get "https://www.storj.io/dcs-satellites": read tcp [2400:6180:0:d0::f70:8001]:42724->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 238.506182ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 42.365412ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/477627029dee8c52 FAIL (2025-11-02 22:38:23 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:52528->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 42.38515ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io) antiblip: 1 out of 15 is failed (threshold=6) [.X.............] duration: 76.158671ms remote: "tcp:[2620:cb:2000::1]:443" target: https://www.storj.io/ trace: https://jaeger.dev.storj.io/trace/477627029dee8c52 FAIL (2025-11-02 22:37:57 UTC) orbiter warning: Get "https://www.storj.io/": read tcp [2400:6180:0:d0::f70:8001]:52528->[2620:cb:2000::1]:443: read: connection reset by peer Duration: 76.177797ms core website

Orbiter Check History

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)

checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)