Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...................................................X..X.....] duration: 3.530004457s trace: https://jaeger.dev.storj.io/trace/41cf535f6e0a50d1 FAIL (2025-11-24 08:13:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.530030898s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [....................................................X..X....] duration: 3.567493907s trace: https://jaeger.dev.storj.io/trace/41cf535f6e0a50d1 FAIL (2025-11-24 08:12:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.567587595s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.442468967s trace: https://jaeger.dev.storj.io/trace/7bc8b7d72c7c544c FAIL (2025-11-24 08:12:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.442500717s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.....................................................X..X...] duration: 3.698771354s trace: https://jaeger.dev.storj.io/trace/41cf535f6e0a50d1 FAIL (2025-11-24 08:11:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.698792577s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.111458699s trace: https://jaeger.dev.storj.io/trace/7bc8b7d72c7c544c FAIL (2025-11-24 08:11:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.111628462s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [......................................................X..X..] duration: 3.483547131s trace: https://jaeger.dev.storj.io/trace/41cf535f6e0a50d1 FAIL (2025-11-24 08:11:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.483567599s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.955539825s trace: https://jaeger.dev.storj.io/trace/7bc8b7d72c7c544c FAIL (2025-11-24 08:11:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.955555584s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.......................................................X..X.] duration: 3.417934335s trace: https://jaeger.dev.storj.io/trace/41cf535f6e0a50d1 FAIL (2025-11-24 08:10:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.417965696s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.695866464s trace: https://jaeger.dev.storj.io/trace/7bc8b7d72c7c544c FAIL (2025-11-24 08:09:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.695881622s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.000990503s trace: https://jaeger.dev.storj.io/trace/7bc8b7d72c7c544c FAIL (2025-11-24 08:09:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001032193s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [........................................................X..X] duration: 5.000505848s trace: https://jaeger.dev.storj.io/trace/41cf535f6e0a50d1 FAIL (2025-11-24 08:09:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000532439s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 3.502699027s trace: https://jaeger.dev.storj.io/trace/4ee1c9aa26aa14fe FAIL (2025-11-24 08:08:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.502725927s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 3.736177527s trace: https://jaeger.dev.storj.io/trace/4ee1c9aa26aa14fe FAIL (2025-11-24 08:07:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.736207584s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.000783689s trace: https://jaeger.dev.storj.io/trace/4ee1c9aa26aa14fe FAIL (2025-11-24 08:06:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.0008681s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.997156997s trace: https://jaeger.dev.storj.io/trace/68b6311f086e60c6 FAIL (2025-11-24 06:05:20 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.997188166s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.986249788s trace: https://jaeger.dev.storj.io/trace/68b6311f086e60c6 FAIL (2025-11-24 06:04:57 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.98627704s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.745910465s trace: https://jaeger.dev.storj.io/trace/599c0302b0cfc801 FAIL (2025-11-24 06:04:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 24 05:23:02.449603695 (357h19m1.001074156s) [upload=314521]: Starting upload Nov 24 05:23:02.449706301 (357h19m1.001176762s) [upload=314521]: Done starting upload Nov 24 05:23:02.449961758 (357h19m1.001432219s) [upload=314521]: Uploading segments... Nov 24 05:23:02.450259785 (357h19m1.001730256s) [upload=314521]: Got next segment. Inline: false Nov 24 05:23:02.450324648 (357h19m1.001795109s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 24 05:23:02.450332814 (357h19m1.001803285s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 24 05:23:02.704332160 (357h19m1.255802621s) [seg_pos={0 0}, upload=314521]: Begin upload segment... Nov 24 05:23:02.704394768 (357h19m1.255865239s) [seg_pos={0 0}, upload=314521]: Done begin upload segment. Nov 24 05:23:02.704436909 (357h19m1.25590736s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.704601662 (357h19m1.256072133s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705249546 (357h19m1.256720017s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705593661 (357h19m1.257064122s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705681198 (357h19m1.257151659s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706230524 (357h19m1.257700985s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706359789 (357h19m1.25783025s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707030456 (357h19m1.258500917s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707439435 (357h19m1.258909896s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707643253 (357h19m1.259113725s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707916354 (357h19m1.259386825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708201466 (357h19m1.259671937s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708344449 (357h19m1.25981491s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708793404 (357h19m1.260263865s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708834722 (357h19m1.260305183s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.710680426 (357h19m1.262150877s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:04.472015490 (357h19m3.023485951s) [upload=314521]: Next returned nil segment Nov 24 05:23:04.472034676 (357h19m3.023505137s) [upload=314521]: Waiting for error group managing segments... Nov 24 05:23:04.788643757 (357h19m3.340114218s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.883924979 (357h19m3.43539544s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.904748416 (357h19m3.456218877s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.905616579 (357h19m3.45708704s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.911750517 (357h19m3.463220978s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.920810467 (357h19m3.472280928s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.924497988 (357h19m3.47596846s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.929369955 (357h19m3.480840416s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.956308374 (357h19m3.507778835s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.982610122 (357h19m3.534080603s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.367340593 (357h19m3.918811064s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.593064333 (357h19m4.144534804s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661884283 (357h19m4.213354744s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661905744 (357h19m4.213376195s) [seg_pos={0 0}, upload=314521]: Segment reached optimal threshold of 13 pieces. Nov 24 05:23:05.662121505 (357h19m4.213591966s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662133698 (357h19m4.213604149s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662158315 (357h19m4.213628776s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection Nov 24 05:23:05.662169947 (357h19m4.213640408s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662326175 (357h19m4.213796626s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662336364 (357h19m4.213806825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662363365 (357h19m4.213833826s) [seg_pos={0 0}, upload=314521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 24 05:23:05.662470149 (357h19m4.21394061s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 24 05:23:05.662478434 (357h19m4.213948895s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 24 05:23:05.964002134 (357h19m4.515472585s) [upload=314521]: Done uploading segments... Nov 24 05:23:05.964027141 (357h19m4.515497613s) [upload=314521]: Upload finished. err: <nil> Duration: 5.745968505s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.806199485s trace: https://jaeger.dev.storj.io/trace/599c0302b0cfc801 FAIL (2025-11-24 06:04:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 24 05:23:02.449603695 (357h19m1.001074156s) [upload=314521]: Starting upload Nov 24 05:23:02.449706301 (357h19m1.001176762s) [upload=314521]: Done starting upload Nov 24 05:23:02.449961758 (357h19m1.001432219s) [upload=314521]: Uploading segments... Nov 24 05:23:02.450259785 (357h19m1.001730256s) [upload=314521]: Got next segment. Inline: false Nov 24 05:23:02.450324648 (357h19m1.001795109s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 24 05:23:02.450332814 (357h19m1.001803285s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 24 05:23:02.704332160 (357h19m1.255802621s) [seg_pos={0 0}, upload=314521]: Begin upload segment... Nov 24 05:23:02.704394768 (357h19m1.255865239s) [seg_pos={0 0}, upload=314521]: Done begin upload segment. Nov 24 05:23:02.704436909 (357h19m1.25590736s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.704601662 (357h19m1.256072133s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705249546 (357h19m1.256720017s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705593661 (357h19m1.257064122s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705681198 (357h19m1.257151659s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706230524 (357h19m1.257700985s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706359789 (357h19m1.25783025s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707030456 (357h19m1.258500917s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707439435 (357h19m1.258909896s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707643253 (357h19m1.259113725s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707916354 (357h19m1.259386825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708201466 (357h19m1.259671937s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708344449 (357h19m1.25981491s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708793404 (357h19m1.260263865s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708834722 (357h19m1.260305183s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.710680426 (357h19m1.262150877s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:04.472015490 (357h19m3.023485951s) [upload=314521]: Next returned nil segment Nov 24 05:23:04.472034676 (357h19m3.023505137s) [upload=314521]: Waiting for error group managing segments... Nov 24 05:23:04.788643757 (357h19m3.340114218s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.883924979 (357h19m3.43539544s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.904748416 (357h19m3.456218877s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.905616579 (357h19m3.45708704s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.911750517 (357h19m3.463220978s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.920810467 (357h19m3.472280928s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.924497988 (357h19m3.47596846s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.929369955 (357h19m3.480840416s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.956308374 (357h19m3.507778835s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.982610122 (357h19m3.534080603s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.367340593 (357h19m3.918811064s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.593064333 (357h19m4.144534804s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661884283 (357h19m4.213354744s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661905744 (357h19m4.213376195s) [seg_pos={0 0}, upload=314521]: Segment reached optimal threshold of 13 pieces. Nov 24 05:23:05.662121505 (357h19m4.213591966s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662133698 (357h19m4.213604149s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662158315 (357h19m4.213628776s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection Nov 24 05:23:05.662169947 (357h19m4.213640408s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662326175 (357h19m4.213796626s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662336364 (357h19m4.213806825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662363365 (357h19m4.213833826s) [seg_pos={0 0}, upload=314521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 24 05:23:05.662470149 (357h19m4.21394061s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 24 05:23:05.662478434 (357h19m4.213948895s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 24 05:23:05.964002134 (357h19m4.515472585s) [upload=314521]: Done uploading segments... Nov 24 05:23:05.964027141 (357h19m4.515497613s) [upload=314521]: Upload finished. err: <nil> Duration: 5.806232727s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.028291314s trace: https://jaeger.dev.storj.io/trace/68b6311f086e60c6 FAIL (2025-11-24 06:03:47 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.028325538s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.85500519s trace: https://jaeger.dev.storj.io/trace/599c0302b0cfc801 FAIL (2025-11-24 06:03:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 24 05:23:02.449603695 (357h19m1.001074156s) [upload=314521]: Starting upload Nov 24 05:23:02.449706301 (357h19m1.001176762s) [upload=314521]: Done starting upload Nov 24 05:23:02.449961758 (357h19m1.001432219s) [upload=314521]: Uploading segments... Nov 24 05:23:02.450259785 (357h19m1.001730256s) [upload=314521]: Got next segment. Inline: false Nov 24 05:23:02.450324648 (357h19m1.001795109s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 24 05:23:02.450332814 (357h19m1.001803285s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 24 05:23:02.704332160 (357h19m1.255802621s) [seg_pos={0 0}, upload=314521]: Begin upload segment... Nov 24 05:23:02.704394768 (357h19m1.255865239s) [seg_pos={0 0}, upload=314521]: Done begin upload segment. Nov 24 05:23:02.704436909 (357h19m1.25590736s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.704601662 (357h19m1.256072133s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705249546 (357h19m1.256720017s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705593661 (357h19m1.257064122s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705681198 (357h19m1.257151659s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706230524 (357h19m1.257700985s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706359789 (357h19m1.25783025s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707030456 (357h19m1.258500917s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707439435 (357h19m1.258909896s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707643253 (357h19m1.259113725s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707916354 (357h19m1.259386825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708201466 (357h19m1.259671937s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708344449 (357h19m1.25981491s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708793404 (357h19m1.260263865s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708834722 (357h19m1.260305183s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.710680426 (357h19m1.262150877s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:04.472015490 (357h19m3.023485951s) [upload=314521]: Next returned nil segment Nov 24 05:23:04.472034676 (357h19m3.023505137s) [upload=314521]: Waiting for error group managing segments... Nov 24 05:23:04.788643757 (357h19m3.340114218s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.883924979 (357h19m3.43539544s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.904748416 (357h19m3.456218877s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.905616579 (357h19m3.45708704s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.911750517 (357h19m3.463220978s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.920810467 (357h19m3.472280928s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.924497988 (357h19m3.47596846s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.929369955 (357h19m3.480840416s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.956308374 (357h19m3.507778835s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.982610122 (357h19m3.534080603s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.367340593 (357h19m3.918811064s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.593064333 (357h19m4.144534804s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661884283 (357h19m4.213354744s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661905744 (357h19m4.213376195s) [seg_pos={0 0}, upload=314521]: Segment reached optimal threshold of 13 pieces. Nov 24 05:23:05.662121505 (357h19m4.213591966s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662133698 (357h19m4.213604149s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662158315 (357h19m4.213628776s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection Nov 24 05:23:05.662169947 (357h19m4.213640408s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662326175 (357h19m4.213796626s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662336364 (357h19m4.213806825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662363365 (357h19m4.213833826s) [seg_pos={0 0}, upload=314521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 24 05:23:05.662470149 (357h19m4.21394061s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 24 05:23:05.662478434 (357h19m4.213948895s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 24 05:23:05.964002134 (357h19m4.515472585s) [upload=314521]: Done uploading segments... Nov 24 05:23:05.964027141 (357h19m4.515497613s) [upload=314521]: Upload finished. err: <nil> Duration: 5.855027291s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.726008457s trace: https://jaeger.dev.storj.io/trace/747a47aa26cdc65f FAIL (2025-11-24 06:03:31 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.726033273s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.00579718s trace: https://jaeger.dev.storj.io/trace/68b6311f086e60c6 FAIL (2025-11-24 06:03:13 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.005824051s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.911066047s trace: https://jaeger.dev.storj.io/trace/747a47aa26cdc65f FAIL (2025-11-24 06:02:51 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.911103017s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.981404331s trace: https://jaeger.dev.storj.io/trace/68b6311f086e60c6 FAIL (2025-11-24 06:02:48 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.981440801s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X..X........................................................] duration: 5.895467617s trace: https://jaeger.dev.storj.io/trace/599c0302b0cfc801 FAIL (2025-11-24 06:02:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 24 05:23:02.449603695 (357h19m1.001074156s) [upload=314521]: Starting upload Nov 24 05:23:02.449706301 (357h19m1.001176762s) [upload=314521]: Done starting upload Nov 24 05:23:02.449961758 (357h19m1.001432219s) [upload=314521]: Uploading segments... Nov 24 05:23:02.450259785 (357h19m1.001730256s) [upload=314521]: Got next segment. Inline: false Nov 24 05:23:02.450324648 (357h19m1.001795109s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 24 05:23:02.450332814 (357h19m1.001803285s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 24 05:23:02.704332160 (357h19m1.255802621s) [seg_pos={0 0}, upload=314521]: Begin upload segment... Nov 24 05:23:02.704394768 (357h19m1.255865239s) [seg_pos={0 0}, upload=314521]: Done begin upload segment. Nov 24 05:23:02.704436909 (357h19m1.25590736s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.704601662 (357h19m1.256072133s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705249546 (357h19m1.256720017s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705593661 (357h19m1.257064122s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705681198 (357h19m1.257151659s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706230524 (357h19m1.257700985s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706359789 (357h19m1.25783025s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707030456 (357h19m1.258500917s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707439435 (357h19m1.258909896s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707643253 (357h19m1.259113725s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707916354 (357h19m1.259386825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708201466 (357h19m1.259671937s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708344449 (357h19m1.25981491s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708793404 (357h19m1.260263865s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708834722 (357h19m1.260305183s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.710680426 (357h19m1.262150877s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:04.472015490 (357h19m3.023485951s) [upload=314521]: Next returned nil segment Nov 24 05:23:04.472034676 (357h19m3.023505137s) [upload=314521]: Waiting for error group managing segments... Nov 24 05:23:04.788643757 (357h19m3.340114218s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.883924979 (357h19m3.43539544s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.904748416 (357h19m3.456218877s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.905616579 (357h19m3.45708704s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.911750517 (357h19m3.463220978s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.920810467 (357h19m3.472280928s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.924497988 (357h19m3.47596846s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.929369955 (357h19m3.480840416s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.956308374 (357h19m3.507778835s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.982610122 (357h19m3.534080603s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.367340593 (357h19m3.918811064s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.593064333 (357h19m4.144534804s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661884283 (357h19m4.213354744s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661905744 (357h19m4.213376195s) [seg_pos={0 0}, upload=314521]: Segment reached optimal threshold of 13 pieces. Nov 24 05:23:05.662121505 (357h19m4.213591966s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662133698 (357h19m4.213604149s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662158315 (357h19m4.213628776s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection Nov 24 05:23:05.662169947 (357h19m4.213640408s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662326175 (357h19m4.213796626s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662336364 (357h19m4.213806825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662363365 (357h19m4.213833826s) [seg_pos={0 0}, upload=314521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 24 05:23:05.662470149 (357h19m4.21394061s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 24 05:23:05.662478434 (357h19m4.213948895s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 24 05:23:05.964002134 (357h19m4.515472585s) [upload=314521]: Done uploading segments... Nov 24 05:23:05.964027141 (357h19m4.515497613s) [upload=314521]: Upload finished. err: <nil> Duration: 5.895494939s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.760069366s trace: https://jaeger.dev.storj.io/trace/747a47aa26cdc65f FAIL (2025-11-24 06:02:11 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.760099954s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XX..X.......................................................] duration: 5.769777862s trace: https://jaeger.dev.storj.io/trace/599c0302b0cfc801 FAIL (2025-11-24 06:01:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 24 05:23:02.449603695 (357h19m1.001074156s) [upload=314521]: Starting upload Nov 24 05:23:02.449706301 (357h19m1.001176762s) [upload=314521]: Done starting upload Nov 24 05:23:02.449961758 (357h19m1.001432219s) [upload=314521]: Uploading segments... Nov 24 05:23:02.450259785 (357h19m1.001730256s) [upload=314521]: Got next segment. Inline: false Nov 24 05:23:02.450324648 (357h19m1.001795109s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 24 05:23:02.450332814 (357h19m1.001803285s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 24 05:23:02.704332160 (357h19m1.255802621s) [seg_pos={0 0}, upload=314521]: Begin upload segment... Nov 24 05:23:02.704394768 (357h19m1.255865239s) [seg_pos={0 0}, upload=314521]: Done begin upload segment. Nov 24 05:23:02.704436909 (357h19m1.25590736s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.704601662 (357h19m1.256072133s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705249546 (357h19m1.256720017s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705593661 (357h19m1.257064122s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705681198 (357h19m1.257151659s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706230524 (357h19m1.257700985s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706359789 (357h19m1.25783025s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707030456 (357h19m1.258500917s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707439435 (357h19m1.258909896s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707643253 (357h19m1.259113725s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707916354 (357h19m1.259386825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708201466 (357h19m1.259671937s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708344449 (357h19m1.25981491s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708793404 (357h19m1.260263865s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708834722 (357h19m1.260305183s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.710680426 (357h19m1.262150877s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:04.472015490 (357h19m3.023485951s) [upload=314521]: Next returned nil segment Nov 24 05:23:04.472034676 (357h19m3.023505137s) [upload=314521]: Waiting for error group managing segments... Nov 24 05:23:04.788643757 (357h19m3.340114218s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.883924979 (357h19m3.43539544s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.904748416 (357h19m3.456218877s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.905616579 (357h19m3.45708704s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.911750517 (357h19m3.463220978s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.920810467 (357h19m3.472280928s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.924497988 (357h19m3.47596846s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.929369955 (357h19m3.480840416s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.956308374 (357h19m3.507778835s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.982610122 (357h19m3.534080603s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.367340593 (357h19m3.918811064s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.593064333 (357h19m4.144534804s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661884283 (357h19m4.213354744s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661905744 (357h19m4.213376195s) [seg_pos={0 0}, upload=314521]: Segment reached optimal threshold of 13 pieces. Nov 24 05:23:05.662121505 (357h19m4.213591966s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662133698 (357h19m4.213604149s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662158315 (357h19m4.213628776s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection Nov 24 05:23:05.662169947 (357h19m4.213640408s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662326175 (357h19m4.213796626s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662336364 (357h19m4.213806825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662363365 (357h19m4.213833826s) [seg_pos={0 0}, upload=314521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 24 05:23:05.662470149 (357h19m4.21394061s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 24 05:23:05.662478434 (357h19m4.213948895s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 24 05:23:05.964002134 (357h19m4.515472585s) [upload=314521]: Done uploading segments... Nov 24 05:23:05.964027141 (357h19m4.515497613s) [upload=314521]: Upload finished. err: <nil> Duration: 5.76980883s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.001644332s trace: https://jaeger.dev.storj.io/trace/68b6311f086e60c6 FAIL (2025-11-24 06:01:38 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.001666424s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.XX..X......................................................] duration: 5.821515277s trace: https://jaeger.dev.storj.io/trace/599c0302b0cfc801 FAIL (2025-11-24 06:01:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 24 05:23:02.449603695 (357h19m1.001074156s) [upload=314521]: Starting upload Nov 24 05:23:02.449706301 (357h19m1.001176762s) [upload=314521]: Done starting upload Nov 24 05:23:02.449961758 (357h19m1.001432219s) [upload=314521]: Uploading segments... Nov 24 05:23:02.450259785 (357h19m1.001730256s) [upload=314521]: Got next segment. Inline: false Nov 24 05:23:02.450324648 (357h19m1.001795109s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 24 05:23:02.450332814 (357h19m1.001803285s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 24 05:23:02.704332160 (357h19m1.255802621s) [seg_pos={0 0}, upload=314521]: Begin upload segment... Nov 24 05:23:02.704394768 (357h19m1.255865239s) [seg_pos={0 0}, upload=314521]: Done begin upload segment. Nov 24 05:23:02.704436909 (357h19m1.25590736s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.704601662 (357h19m1.256072133s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705249546 (357h19m1.256720017s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705593661 (357h19m1.257064122s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705681198 (357h19m1.257151659s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706230524 (357h19m1.257700985s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706359789 (357h19m1.25783025s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707030456 (357h19m1.258500917s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707439435 (357h19m1.258909896s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707643253 (357h19m1.259113725s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707916354 (357h19m1.259386825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708201466 (357h19m1.259671937s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708344449 (357h19m1.25981491s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708793404 (357h19m1.260263865s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708834722 (357h19m1.260305183s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.710680426 (357h19m1.262150877s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:04.472015490 (357h19m3.023485951s) [upload=314521]: Next returned nil segment Nov 24 05:23:04.472034676 (357h19m3.023505137s) [upload=314521]: Waiting for error group managing segments... Nov 24 05:23:04.788643757 (357h19m3.340114218s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.883924979 (357h19m3.43539544s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.904748416 (357h19m3.456218877s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.905616579 (357h19m3.45708704s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.911750517 (357h19m3.463220978s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.920810467 (357h19m3.472280928s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.924497988 (357h19m3.47596846s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.929369955 (357h19m3.480840416s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.956308374 (357h19m3.507778835s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.982610122 (357h19m3.534080603s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.367340593 (357h19m3.918811064s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.593064333 (357h19m4.144534804s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661884283 (357h19m4.213354744s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661905744 (357h19m4.213376195s) [seg_pos={0 0}, upload=314521]: Segment reached optimal threshold of 13 pieces. Nov 24 05:23:05.662121505 (357h19m4.213591966s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662133698 (357h19m4.213604149s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662158315 (357h19m4.213628776s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection Nov 24 05:23:05.662169947 (357h19m4.213640408s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662326175 (357h19m4.213796626s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662336364 (357h19m4.213806825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662363365 (357h19m4.213833826s) [seg_pos={0 0}, upload=314521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 24 05:23:05.662470149 (357h19m4.21394061s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 24 05:23:05.662478434 (357h19m4.213948895s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 24 05:23:05.964002134 (357h19m4.515472585s) [upload=314521]: Done uploading segments... Nov 24 05:23:05.964027141 (357h19m4.515497613s) [upload=314521]: Upload finished. err: <nil> Duration: 5.821544344s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.098422346s trace: https://jaeger.dev.storj.io/trace/747a47aa26cdc65f FAIL (2025-11-24 06:01:05 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 6.098474635s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.994891795s trace: https://jaeger.dev.storj.io/trace/68b6311f086e60c6 FAIL (2025-11-24 06:00:53 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.994920438s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..XX..X.....................................................] duration: 6.013240547s trace: https://jaeger.dev.storj.io/trace/599c0302b0cfc801 FAIL (2025-11-24 06:00:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 24 05:23:02.449603695 (357h19m1.001074156s) [upload=314521]: Starting upload Nov 24 05:23:02.449706301 (357h19m1.001176762s) [upload=314521]: Done starting upload Nov 24 05:23:02.449961758 (357h19m1.001432219s) [upload=314521]: Uploading segments... Nov 24 05:23:02.450259785 (357h19m1.001730256s) [upload=314521]: Got next segment. Inline: false Nov 24 05:23:02.450324648 (357h19m1.001795109s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 24 05:23:02.450332814 (357h19m1.001803285s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 24 05:23:02.704332160 (357h19m1.255802621s) [seg_pos={0 0}, upload=314521]: Begin upload segment... Nov 24 05:23:02.704394768 (357h19m1.255865239s) [seg_pos={0 0}, upload=314521]: Done begin upload segment. Nov 24 05:23:02.704436909 (357h19m1.25590736s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.704601662 (357h19m1.256072133s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705249546 (357h19m1.256720017s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705593661 (357h19m1.257064122s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705681198 (357h19m1.257151659s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706230524 (357h19m1.257700985s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706359789 (357h19m1.25783025s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707030456 (357h19m1.258500917s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707439435 (357h19m1.258909896s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707643253 (357h19m1.259113725s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707916354 (357h19m1.259386825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708201466 (357h19m1.259671937s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708344449 (357h19m1.25981491s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708793404 (357h19m1.260263865s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708834722 (357h19m1.260305183s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.710680426 (357h19m1.262150877s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:04.472015490 (357h19m3.023485951s) [upload=314521]: Next returned nil segment Nov 24 05:23:04.472034676 (357h19m3.023505137s) [upload=314521]: Waiting for error group managing segments... Nov 24 05:23:04.788643757 (357h19m3.340114218s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.883924979 (357h19m3.43539544s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.904748416 (357h19m3.456218877s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.905616579 (357h19m3.45708704s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.911750517 (357h19m3.463220978s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.920810467 (357h19m3.472280928s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.924497988 (357h19m3.47596846s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.929369955 (357h19m3.480840416s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.956308374 (357h19m3.507778835s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.982610122 (357h19m3.534080603s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.367340593 (357h19m3.918811064s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.593064333 (357h19m4.144534804s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661884283 (357h19m4.213354744s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661905744 (357h19m4.213376195s) [seg_pos={0 0}, upload=314521]: Segment reached optimal threshold of 13 pieces. Nov 24 05:23:05.662121505 (357h19m4.213591966s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662133698 (357h19m4.213604149s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662158315 (357h19m4.213628776s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection Nov 24 05:23:05.662169947 (357h19m4.213640408s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662326175 (357h19m4.213796626s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662336364 (357h19m4.213806825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662363365 (357h19m4.213833826s) [seg_pos={0 0}, upload=314521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 24 05:23:05.662470149 (357h19m4.21394061s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 24 05:23:05.662478434 (357h19m4.213948895s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 24 05:23:05.964002134 (357h19m4.515472585s) [upload=314521]: Done uploading segments... Nov 24 05:23:05.964027141 (357h19m4.515497613s) [upload=314521]: Upload finished. err: <nil> Duration: 6.013274621s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.933453808s trace: https://jaeger.dev.storj.io/trace/747a47aa26cdc65f FAIL (2025-11-24 06:00:37 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.933482123s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...XX..X....................................................] duration: 6.643611911s trace: https://jaeger.dev.storj.io/trace/599c0302b0cfc801 FAIL (2025-11-24 06:00:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 24 05:23:02.449603695 (357h19m1.001074156s) [upload=314521]: Starting upload Nov 24 05:23:02.449706301 (357h19m1.001176762s) [upload=314521]: Done starting upload Nov 24 05:23:02.449961758 (357h19m1.001432219s) [upload=314521]: Uploading segments... Nov 24 05:23:02.450259785 (357h19m1.001730256s) [upload=314521]: Got next segment. Inline: false Nov 24 05:23:02.450324648 (357h19m1.001795109s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 24 05:23:02.450332814 (357h19m1.001803285s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 24 05:23:02.704332160 (357h19m1.255802621s) [seg_pos={0 0}, upload=314521]: Begin upload segment... Nov 24 05:23:02.704394768 (357h19m1.255865239s) [seg_pos={0 0}, upload=314521]: Done begin upload segment. Nov 24 05:23:02.704436909 (357h19m1.25590736s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.704601662 (357h19m1.256072133s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705249546 (357h19m1.256720017s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705593661 (357h19m1.257064122s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705681198 (357h19m1.257151659s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706230524 (357h19m1.257700985s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706359789 (357h19m1.25783025s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707030456 (357h19m1.258500917s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707439435 (357h19m1.258909896s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707643253 (357h19m1.259113725s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707916354 (357h19m1.259386825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708201466 (357h19m1.259671937s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708344449 (357h19m1.25981491s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708793404 (357h19m1.260263865s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708834722 (357h19m1.260305183s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.710680426 (357h19m1.262150877s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:04.472015490 (357h19m3.023485951s) [upload=314521]: Next returned nil segment Nov 24 05:23:04.472034676 (357h19m3.023505137s) [upload=314521]: Waiting for error group managing segments... Nov 24 05:23:04.788643757 (357h19m3.340114218s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.883924979 (357h19m3.43539544s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.904748416 (357h19m3.456218877s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.905616579 (357h19m3.45708704s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.911750517 (357h19m3.463220978s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.920810467 (357h19m3.472280928s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.924497988 (357h19m3.47596846s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.929369955 (357h19m3.480840416s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.956308374 (357h19m3.507778835s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.982610122 (357h19m3.534080603s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.367340593 (357h19m3.918811064s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.593064333 (357h19m4.144534804s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661884283 (357h19m4.213354744s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661905744 (357h19m4.213376195s) [seg_pos={0 0}, upload=314521]: Segment reached optimal threshold of 13 pieces. Nov 24 05:23:05.662121505 (357h19m4.213591966s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662133698 (357h19m4.213604149s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662158315 (357h19m4.213628776s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection Nov 24 05:23:05.662169947 (357h19m4.213640408s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662326175 (357h19m4.213796626s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662336364 (357h19m4.213806825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662363365 (357h19m4.213833826s) [seg_pos={0 0}, upload=314521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 24 05:23:05.662470149 (357h19m4.21394061s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 24 05:23:05.662478434 (357h19m4.213948895s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 24 05:23:05.964002134 (357h19m4.515472585s) [upload=314521]: Done uploading segments... Nov 24 05:23:05.964027141 (357h19m4.515497613s) [upload=314521]: Upload finished. err: <nil> Duration: 6.643645314s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 1.942140417s trace: https://jaeger.dev.storj.io/trace/68b6311f086e60c6 FAIL (2025-11-24 05:59:56 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.942167408s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.795266878s trace: https://jaeger.dev.storj.io/trace/747a47aa26cdc65f FAIL (2025-11-24 05:59:45 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.795288227s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....XX..X...................................................] duration: 5.981207239s trace: https://jaeger.dev.storj.io/trace/599c0302b0cfc801 FAIL (2025-11-24 05:59:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 24 05:23:02.449603695 (357h19m1.001074156s) [upload=314521]: Starting upload Nov 24 05:23:02.449706301 (357h19m1.001176762s) [upload=314521]: Done starting upload Nov 24 05:23:02.449961758 (357h19m1.001432219s) [upload=314521]: Uploading segments... Nov 24 05:23:02.450259785 (357h19m1.001730256s) [upload=314521]: Got next segment. Inline: false Nov 24 05:23:02.450324648 (357h19m1.001795109s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 24 05:23:02.450332814 (357h19m1.001803285s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 24 05:23:02.704332160 (357h19m1.255802621s) [seg_pos={0 0}, upload=314521]: Begin upload segment... Nov 24 05:23:02.704394768 (357h19m1.255865239s) [seg_pos={0 0}, upload=314521]: Done begin upload segment. Nov 24 05:23:02.704436909 (357h19m1.25590736s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.704601662 (357h19m1.256072133s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705249546 (357h19m1.256720017s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705593661 (357h19m1.257064122s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705681198 (357h19m1.257151659s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706230524 (357h19m1.257700985s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706359789 (357h19m1.25783025s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707030456 (357h19m1.258500917s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707439435 (357h19m1.258909896s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707643253 (357h19m1.259113725s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707916354 (357h19m1.259386825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708201466 (357h19m1.259671937s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708344449 (357h19m1.25981491s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708793404 (357h19m1.260263865s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708834722 (357h19m1.260305183s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.710680426 (357h19m1.262150877s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:04.472015490 (357h19m3.023485951s) [upload=314521]: Next returned nil segment Nov 24 05:23:04.472034676 (357h19m3.023505137s) [upload=314521]: Waiting for error group managing segments... Nov 24 05:23:04.788643757 (357h19m3.340114218s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.883924979 (357h19m3.43539544s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.904748416 (357h19m3.456218877s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.905616579 (357h19m3.45708704s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.911750517 (357h19m3.463220978s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.920810467 (357h19m3.472280928s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.924497988 (357h19m3.47596846s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.929369955 (357h19m3.480840416s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.956308374 (357h19m3.507778835s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.982610122 (357h19m3.534080603s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.367340593 (357h19m3.918811064s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.593064333 (357h19m4.144534804s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661884283 (357h19m4.213354744s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661905744 (357h19m4.213376195s) [seg_pos={0 0}, upload=314521]: Segment reached optimal threshold of 13 pieces. Nov 24 05:23:05.662121505 (357h19m4.213591966s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662133698 (357h19m4.213604149s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662158315 (357h19m4.213628776s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection Nov 24 05:23:05.662169947 (357h19m4.213640408s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662326175 (357h19m4.213796626s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662336364 (357h19m4.213806825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662363365 (357h19m4.213833826s) [seg_pos={0 0}, upload=314521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 24 05:23:05.662470149 (357h19m4.21394061s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 24 05:23:05.662478434 (357h19m4.213948895s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 24 05:23:05.964002134 (357h19m4.515472585s) [upload=314521]: Done uploading segments... Nov 24 05:23:05.964027141 (357h19m4.515497613s) [upload=314521]: Upload finished. err: <nil> Duration: 5.981236958s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 1.970107441s trace: https://jaeger.dev.storj.io/trace/68b6311f086e60c6 FAIL (2025-11-24 05:59:35 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.970128491s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 2.06472572s trace: https://jaeger.dev.storj.io/trace/68b6311f086e60c6 FAIL (2025-11-24 05:59:19 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.064744365s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.838103375s trace: https://jaeger.dev.storj.io/trace/747a47aa26cdc65f FAIL (2025-11-24 05:58:55 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.838134295s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....XX..X..................................................] duration: 5.787804276s trace: https://jaeger.dev.storj.io/trace/599c0302b0cfc801 FAIL (2025-11-24 05:58:46 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 24 05:23:02.449603695 (357h19m1.001074156s) [upload=314521]: Starting upload Nov 24 05:23:02.449706301 (357h19m1.001176762s) [upload=314521]: Done starting upload Nov 24 05:23:02.449961758 (357h19m1.001432219s) [upload=314521]: Uploading segments... Nov 24 05:23:02.450259785 (357h19m1.001730256s) [upload=314521]: Got next segment. Inline: false Nov 24 05:23:02.450324648 (357h19m1.001795109s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 24 05:23:02.450332814 (357h19m1.001803285s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 24 05:23:02.704332160 (357h19m1.255802621s) [seg_pos={0 0}, upload=314521]: Begin upload segment... Nov 24 05:23:02.704394768 (357h19m1.255865239s) [seg_pos={0 0}, upload=314521]: Done begin upload segment. Nov 24 05:23:02.704436909 (357h19m1.25590736s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.704601662 (357h19m1.256072133s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705249546 (357h19m1.256720017s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705593661 (357h19m1.257064122s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705681198 (357h19m1.257151659s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706230524 (357h19m1.257700985s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706359789 (357h19m1.25783025s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707030456 (357h19m1.258500917s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707439435 (357h19m1.258909896s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707643253 (357h19m1.259113725s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707916354 (357h19m1.259386825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708201466 (357h19m1.259671937s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708344449 (357h19m1.25981491s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708793404 (357h19m1.260263865s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708834722 (357h19m1.260305183s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.710680426 (357h19m1.262150877s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:04.472015490 (357h19m3.023485951s) [upload=314521]: Next returned nil segment Nov 24 05:23:04.472034676 (357h19m3.023505137s) [upload=314521]: Waiting for error group managing segments... Nov 24 05:23:04.788643757 (357h19m3.340114218s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.883924979 (357h19m3.43539544s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.904748416 (357h19m3.456218877s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.905616579 (357h19m3.45708704s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.911750517 (357h19m3.463220978s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.920810467 (357h19m3.472280928s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.924497988 (357h19m3.47596846s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.929369955 (357h19m3.480840416s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.956308374 (357h19m3.507778835s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.982610122 (357h19m3.534080603s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.367340593 (357h19m3.918811064s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.593064333 (357h19m4.144534804s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661884283 (357h19m4.213354744s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661905744 (357h19m4.213376195s) [seg_pos={0 0}, upload=314521]: Segment reached optimal threshold of 13 pieces. Nov 24 05:23:05.662121505 (357h19m4.213591966s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662133698 (357h19m4.213604149s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662158315 (357h19m4.213628776s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection Nov 24 05:23:05.662169947 (357h19m4.213640408s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662326175 (357h19m4.213796626s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662336364 (357h19m4.213806825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662363365 (357h19m4.213833826s) [seg_pos={0 0}, upload=314521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 24 05:23:05.662470149 (357h19m4.21394061s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 24 05:23:05.662478434 (357h19m4.213948895s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 24 05:23:05.964002134 (357h19m4.515472585s) [upload=314521]: Done uploading segments... Nov 24 05:23:05.964027141 (357h19m4.515497613s) [upload=314521]: Upload finished. err: <nil> Duration: 5.787834535s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 2.007098097s trace: https://jaeger.dev.storj.io/trace/68b6311f086e60c6 FAIL (2025-11-24 05:58:18 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.007132081s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......XX..X.................................................] duration: 5.642307128s trace: https://jaeger.dev.storj.io/trace/599c0302b0cfc801 FAIL (2025-11-24 05:57:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 24 05:23:02.449603695 (357h19m1.001074156s) [upload=314521]: Starting upload Nov 24 05:23:02.449706301 (357h19m1.001176762s) [upload=314521]: Done starting upload Nov 24 05:23:02.449961758 (357h19m1.001432219s) [upload=314521]: Uploading segments... Nov 24 05:23:02.450259785 (357h19m1.001730256s) [upload=314521]: Got next segment. Inline: false Nov 24 05:23:02.450324648 (357h19m1.001795109s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 24 05:23:02.450332814 (357h19m1.001803285s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 24 05:23:02.704332160 (357h19m1.255802621s) [seg_pos={0 0}, upload=314521]: Begin upload segment... Nov 24 05:23:02.704394768 (357h19m1.255865239s) [seg_pos={0 0}, upload=314521]: Done begin upload segment. Nov 24 05:23:02.704436909 (357h19m1.25590736s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.704601662 (357h19m1.256072133s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705249546 (357h19m1.256720017s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705593661 (357h19m1.257064122s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705681198 (357h19m1.257151659s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706230524 (357h19m1.257700985s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706359789 (357h19m1.25783025s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707030456 (357h19m1.258500917s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707439435 (357h19m1.258909896s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707643253 (357h19m1.259113725s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707916354 (357h19m1.259386825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708201466 (357h19m1.259671937s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708344449 (357h19m1.25981491s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708793404 (357h19m1.260263865s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708834722 (357h19m1.260305183s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.710680426 (357h19m1.262150877s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:04.472015490 (357h19m3.023485951s) [upload=314521]: Next returned nil segment Nov 24 05:23:04.472034676 (357h19m3.023505137s) [upload=314521]: Waiting for error group managing segments... Nov 24 05:23:04.788643757 (357h19m3.340114218s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.883924979 (357h19m3.43539544s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.904748416 (357h19m3.456218877s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.905616579 (357h19m3.45708704s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.911750517 (357h19m3.463220978s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.920810467 (357h19m3.472280928s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.924497988 (357h19m3.47596846s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.929369955 (357h19m3.480840416s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.956308374 (357h19m3.507778835s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.982610122 (357h19m3.534080603s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.367340593 (357h19m3.918811064s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.593064333 (357h19m4.144534804s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661884283 (357h19m4.213354744s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661905744 (357h19m4.213376195s) [seg_pos={0 0}, upload=314521]: Segment reached optimal threshold of 13 pieces. Nov 24 05:23:05.662121505 (357h19m4.213591966s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662133698 (357h19m4.213604149s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662158315 (357h19m4.213628776s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection Nov 24 05:23:05.662169947 (357h19m4.213640408s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662326175 (357h19m4.213796626s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662336364 (357h19m4.213806825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662363365 (357h19m4.213833826s) [seg_pos={0 0}, upload=314521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 24 05:23:05.662470149 (357h19m4.21394061s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 24 05:23:05.662478434 (357h19m4.213948895s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 24 05:23:05.964002134 (357h19m4.515472585s) [upload=314521]: Done uploading segments... Nov 24 05:23:05.964027141 (357h19m4.515497613s) [upload=314521]: Upload finished. err: <nil> Duration: 5.642328018s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.691591142s trace: https://jaeger.dev.storj.io/trace/747a47aa26cdc65f FAIL (2025-11-24 05:57:40 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.691623744s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 1.991143878s trace: https://jaeger.dev.storj.io/trace/68b6311f086e60c6 FAIL (2025-11-24 05:57:30 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.99117653s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.792540666s trace: https://jaeger.dev.storj.io/trace/747a47aa26cdc65f FAIL (2025-11-24 05:57:21 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.792577876s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.70350341s trace: https://jaeger.dev.storj.io/trace/747a47aa26cdc65f FAIL (2025-11-24 05:56:54 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.703531913s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......XX..X................................................] duration: 5.941700579s trace: https://jaeger.dev.storj.io/trace/599c0302b0cfc801 FAIL (2025-11-24 05:56:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Nov 24 05:23:02.449603695 (357h19m1.001074156s) [upload=314521]: Starting upload Nov 24 05:23:02.449706301 (357h19m1.001176762s) [upload=314521]: Done starting upload Nov 24 05:23:02.449961758 (357h19m1.001432219s) [upload=314521]: Uploading segments... Nov 24 05:23:02.450259785 (357h19m1.001730256s) [upload=314521]: Got next segment. Inline: false Nov 24 05:23:02.450324648 (357h19m1.001795109s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Nov 24 05:23:02.450332814 (357h19m1.001803285s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Nov 24 05:23:02.704332160 (357h19m1.255802621s) [seg_pos={0 0}, upload=314521]: Begin upload segment... Nov 24 05:23:02.704394768 (357h19m1.255865239s) [seg_pos={0 0}, upload=314521]: Done begin upload segment. Nov 24 05:23:02.704436909 (357h19m1.25590736s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.704601662 (357h19m1.256072133s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705249546 (357h19m1.256720017s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705593661 (357h19m1.257064122s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.705681198 (357h19m1.257151659s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706230524 (357h19m1.257700985s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.706359789 (357h19m1.25783025s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707030456 (357h19m1.258500917s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707439435 (357h19m1.258909896s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707643253 (357h19m1.259113725s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.707916354 (357h19m1.259386825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708201466 (357h19m1.259671937s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708344449 (357h19m1.25981491s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708793404 (357h19m1.260263865s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.708834722 (357h19m1.260305183s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:02.710680426 (357h19m1.262150877s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Uploading piece... Nov 24 05:23:04.472015490 (357h19m3.023485951s) [upload=314521]: Next returned nil segment Nov 24 05:23:04.472034676 (357h19m3.023505137s) [upload=314521]: Waiting for error group managing segments... Nov 24 05:23:04.788643757 (357h19m3.340114218s) [noise=true , address=77.90.31.15:4697 , piece_id=M7IXD6EXW5CJMEHHFTWIKL3BOTGTO46FHXMG4VSYUA2JSAVNNOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.883924979 (357h19m3.43539544s) [noise=true , address=45.91.116.9:4263 , piece_id=X7DXQMI7BARFP3HXG37RDMMEZX2UENQ3V5PMZVHQP3QPB22UOOUQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.904748416 (357h19m3.456218877s) [noise=true , address=145.79.227.24:4583 , piece_id=AQ4RPI4YSZ4RWG7LZJ446BQE4F3JLE7EV5KMBT5NAOLIPBOTTKVA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.905616579 (357h19m3.45708704s) [noise=true , address=145.79.227.21:4474 , piece_id=QZJG7QREDA4YB7L3Z5LRZO4JEGSON2EDMWCXERPYVYUQDOUN7ZPA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.911750517 (357h19m3.463220978s) [noise=true , address=77.90.31.13:4626 , piece_id=WCO3L3P7VYS3NUL2DHZUDC3NLEMSP7KYTS7VG7I6PGDA2YLWCSCA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.920810467 (357h19m3.472280928s) [noise=true , address=145.79.227.14:4838 , piece_id=OWWFG2DWVP55FWHD3CLG3PJEBBEIDFFO3R3EBRW7P27RU3OF53OQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.924497988 (357h19m3.47596846s) [noise=true , address=145.79.227.17:4971 , piece_id=P43EREHGSJCMHDC5ZULWYHNPW22SIGWOTEYYGCWV3LYRZM34M4CQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.929369955 (357h19m3.480840416s) [noise=true , address=77.90.31.14:4661 , piece_id=CNCRYTPG75Z6HK5CXSNI3NKDOGJVF5NECN5UZTT2VPMJ56EPYNJQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.956308374 (357h19m3.507778835s) [noise=true , address=145.79.227.8:4331 , piece_id=CT3RJJ73AU7M7XAJK3DC54H4EXHIJLKU6OXCR7MCMYEBWKKSWE5Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:04.982610122 (357h19m3.534080603s) [noise=true , address=77.90.31.21:4879 , piece_id=G5O2UHPQZ75I63YCRYAJEWMO7HSCW2RH3QVHM4DUD37A5SOVBXYQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.367340593 (357h19m3.918811064s) [noise=true , address=145.79.227.9:4365 , piece_id=J4F4MS5R3IRZRVFHDM6V2WGGFG4VW62DRMS6D4GWGW5CNFI35AMA, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.593064333 (357h19m4.144534804s) [noise=true , address=145.79.227.31:4797 , piece_id=EDWIDLR7KLDQNG6BKWFTXOUOJLKJKSM7ILTINHY5F3KYLTO3FDBQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661884283 (357h19m4.213354744s) [noise=true , address=145.79.227.24:4582 , piece_id=OCZFWWWK2ECNRISYQK5ILWJZSDDPZKWK7HRBGPNBMDHTA4OJDQQQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: <nil> Nov 24 05:23:05.661905744 (357h19m4.213376195s) [seg_pos={0 0}, upload=314521]: Segment reached optimal threshold of 13 pieces. Nov 24 05:23:05.662121505 (357h19m4.213591966s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CJCyYubAEcM8Tf9daXGiMaazEQRczgPWF8G2Xmox3SE9AWueQ): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662133698 (357h19m4.213604149s) [noise=true , address=145.79.227.26:4653 , piece_id=ZK63CKT3NLXRYREB2QXKUL7HLWVIODAEEIKSUQCHQTIO2DXICB6Q, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662158315 (357h19m4.213628776s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x): protocol: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37484->145.79.227.16:4909: use of closed network connection Nov 24 05:23:05.662169947 (357h19m4.213640408s) [noise=true , address=145.79.227.16:4909 , piece_id=YM5K4ALDMGNRRMOVAVIRVS2TC7IAG5KEJW2BZUZNCS4XHSWR7YCQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662326175 (357h19m4.213796626s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3): protocol: expected piece hash; context canceled; EOF Nov 24 05:23:05.662336364 (357h19m4.213806825s) [noise=true , address=145.79.227.28:4727 , piece_id=MKLU2V2DXZXGKJDCYRAVN42OHIQK4ZCNHOUG4MNW2JMTJP6UK5YQ, seg_pos={0 0}, upload=314521]: Long tail cancellation due to optimal threshold Nov 24 05:23:05.662363365 (357h19m4.213833826s) [seg_pos={0 0}, upload=314521]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Nov 24 05:23:05.662470149 (357h19m4.21394061s) [upload=314521]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 24 05:23:05.662478434 (357h19m4.213948895s) [upload=314521]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 24 05:23:05.964002134 (357h19m4.515472585s) [upload=314521]: Done uploading segments... Nov 24 05:23:05.964027141 (357h19m4.515497613s) [upload=314521]: Upload finished. err: <nil> Duration: 5.941731527s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 1.944589847s trace: https://jaeger.dev.storj.io/trace/68b6311f086e60c6 FAIL (2025-11-24 05:56:44 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 1.94461802s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.933185974s trace: https://jaeger.dev.storj.io/trace/747a47aa26cdc65f FAIL (2025-11-24 05:56:21 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 5.933216924s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)