Orbiter History

Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 865880 reports (Page 11676 of 17318)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 1.997184067s trace: https://jaeger.dev.storj.io/trace/1ea7632defe62c27 FAIL (2025-12-08 22:14:45 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.997208093s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.037379453s trace: https://jaeger.dev.storj.io/trace/51c602d400b15cea FAIL (2025-12-08 22:14:44 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.037421704s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 17.928046925s trace: https://jaeger.dev.storj.io/trace/1ea7632defe62c27 FAIL (2025-12-08 22:14:42 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 17.928099827s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.75769368s trace: https://jaeger.dev.storj.io/trace/1aa6e2916420d374 FAIL (2025-12-08 22:14:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.757720751s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.X....X.X..................................X..X.........X...] duration: 5.694509939s trace: https://jaeger.dev.storj.io/trace/1e577f05655b28db FAIL (2025-12-08 22:14:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.694524758s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [....X...X.....XXX............................X.X............] duration: 3.519012893s trace: https://jaeger.dev.storj.io/trace/6f4f8cb243e59639 FAIL (2025-12-08 22:14:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.51903805s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.836748877s trace: https://jaeger.dev.storj.io/trace/1aa6e2916420d374 FAIL (2025-12-08 22:13:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.83677692s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................X...............X..............] duration: 5.770362304s trace: https://jaeger.dev.storj.io/trace/72871f7c52601eaf FAIL (2025-12-08 22:13:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== _pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917385480 (78h34m23.788468516s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917994611 (78h34m23.789077666s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918445960 (78h34m23.789528995s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918888933 (78h34m23.789971978s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919333861 (78h34m23.790416906s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919736458 (78h34m23.790819503s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920148312 (78h34m23.791231347s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920546540 (78h34m23.791629585s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.921942007 (78h34m23.793025052s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.923813691 (78h34m23.794896736s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.925353112 (78h34m23.796436157s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.927009226 (78h34m23.798092261s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.928697040 (78h34m23.799780085s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.931092261 (78h34m23.802175306s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.932748024 (78h34m23.803831069s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.933733821 (78h34m23.804816856s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.935468164 (78h34m23.806551209s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:25.369562765 (78h34m25.24064581s) [upload=68635]: Next returned nil segment Dec 8 22:03:25.369578857 (78h34m25.240661892s) [upload=68635]: Waiting for error group managing segments... Dec 8 22:03:25.775620953 (78h34m25.646703999s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.798803277 (78h34m25.669886322s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.801642202 (78h34m25.672725237s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.802057232 (78h34m25.673140267s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816132514 (78h34m25.687215559s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816446331 (78h34m25.687529376s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.869308801 (78h34m25.740391846s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.873476507 (78h34m25.744559542s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.907465318 (78h34m25.778548354s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.926737626 (78h34m25.797820671s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.928011191 (78h34m25.799094226s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.930540317 (78h34m25.801623362s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.954520891 (78h34m25.825603926s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001264168 (78h34m25.872347203s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001283154 (78h34m25.872366189s) [seg_pos={0 0}, upload=68635]: Segment reached optimal threshold of 14 pieces. Dec 8 22:03:26.001442167 (78h34m25.872525202s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001448459 (78h34m25.872531494s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001703023 (78h34m25.872786068s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123uv49k8LraXJfYQg6KRQthrYs1Egc4c3eZWm5JPvDatFbykm8): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001710758 (78h34m25.872793803s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001867176 (78h34m25.872950231s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Myourwn6UnUxVkxzBdRJApeAGNrZzjrgLgzAfki6x9JZ1Fy17): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001872526 (78h34m25.872955581s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002060293 (78h34m25.873143328s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHCVWGoutBZ3Du4xTCCJGALvrAZwB7cmC1vsRGj5LcwyfW8Qe): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.002066315 (78h34m25.873149351s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002085732 (78h34m25.873168767s) [seg_pos={0 0}, upload=68635]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 22:03:26.002156887 (78h34m25.873239922s) [upload=68635]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 22:03:26.002161526 (78h34m25.873244561s) [upload=68635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 22:04:50.380087807 (78h35m50.251170842s) [upload=68635]: Done uploading segments... Dec 8 22:04:50.380137331 (78h35m50.251220376s) [upload=68635]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.770381329s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.013444127s trace: https://jaeger.dev.storj.io/trace/1ea7632defe62c27 FAIL (2025-12-08 22:13:41 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.013473124s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 4.842177593s trace: https://jaeger.dev.storj.io/trace/51c602d400b15cea FAIL (2025-12-08 22:13:39 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.842203914s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [..X....X.X..................................X..X.........X..] duration: 5.891279382s trace: https://jaeger.dev.storj.io/trace/1e577f05655b28db FAIL (2025-12-08 22:13:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.891338405s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [.....X...X.....XXX............................X.X...........] duration: 3.537792822s trace: https://jaeger.dev.storj.io/trace/6f4f8cb243e59639 FAIL (2025-12-08 22:13:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.5378182s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.710862636s trace: https://jaeger.dev.storj.io/trace/1aa6e2916420d374 FAIL (2025-12-08 22:12:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.710888495s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................X...............X.............] duration: 5.728450278s trace: https://jaeger.dev.storj.io/trace/72871f7c52601eaf FAIL (2025-12-08 22:12:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== _pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917385480 (78h34m23.788468516s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917994611 (78h34m23.789077666s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918445960 (78h34m23.789528995s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918888933 (78h34m23.789971978s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919333861 (78h34m23.790416906s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919736458 (78h34m23.790819503s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920148312 (78h34m23.791231347s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920546540 (78h34m23.791629585s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.921942007 (78h34m23.793025052s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.923813691 (78h34m23.794896736s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.925353112 (78h34m23.796436157s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.927009226 (78h34m23.798092261s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.928697040 (78h34m23.799780085s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.931092261 (78h34m23.802175306s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.932748024 (78h34m23.803831069s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.933733821 (78h34m23.804816856s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.935468164 (78h34m23.806551209s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:25.369562765 (78h34m25.24064581s) [upload=68635]: Next returned nil segment Dec 8 22:03:25.369578857 (78h34m25.240661892s) [upload=68635]: Waiting for error group managing segments... Dec 8 22:03:25.775620953 (78h34m25.646703999s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.798803277 (78h34m25.669886322s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.801642202 (78h34m25.672725237s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.802057232 (78h34m25.673140267s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816132514 (78h34m25.687215559s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816446331 (78h34m25.687529376s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.869308801 (78h34m25.740391846s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.873476507 (78h34m25.744559542s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.907465318 (78h34m25.778548354s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.926737626 (78h34m25.797820671s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.928011191 (78h34m25.799094226s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.930540317 (78h34m25.801623362s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.954520891 (78h34m25.825603926s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001264168 (78h34m25.872347203s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001283154 (78h34m25.872366189s) [seg_pos={0 0}, upload=68635]: Segment reached optimal threshold of 14 pieces. Dec 8 22:03:26.001442167 (78h34m25.872525202s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001448459 (78h34m25.872531494s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001703023 (78h34m25.872786068s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123uv49k8LraXJfYQg6KRQthrYs1Egc4c3eZWm5JPvDatFbykm8): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001710758 (78h34m25.872793803s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001867176 (78h34m25.872950231s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Myourwn6UnUxVkxzBdRJApeAGNrZzjrgLgzAfki6x9JZ1Fy17): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001872526 (78h34m25.872955581s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002060293 (78h34m25.873143328s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHCVWGoutBZ3Du4xTCCJGALvrAZwB7cmC1vsRGj5LcwyfW8Qe): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.002066315 (78h34m25.873149351s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002085732 (78h34m25.873168767s) [seg_pos={0 0}, upload=68635]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 22:03:26.002156887 (78h34m25.873239922s) [upload=68635]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 22:03:26.002161526 (78h34m25.873244561s) [upload=68635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 22:04:50.380087807 (78h35m50.251170842s) [upload=68635]: Done uploading segments... Dec 8 22:04:50.380137331 (78h35m50.251220376s) [upload=68635]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.728480807s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 2.047649762s trace: https://jaeger.dev.storj.io/trace/1ea7632defe62c27 FAIL (2025-12-08 22:12:42 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.047713825s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 4.95566726s trace: https://jaeger.dev.storj.io/trace/51c602d400b15cea FAIL (2025-12-08 22:12:42 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.955686276s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [X..X....X.X..................................X..X.........X.] duration: 5.69624874s trace: https://jaeger.dev.storj.io/trace/1e577f05655b28db FAIL (2025-12-08 22:12:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.696264701s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.043842069s trace: https://jaeger.dev.storj.io/trace/1aa6e2916420d374 FAIL (2025-12-08 22:12:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.043865543s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [.X..X....X.X..................................X..X.........X] duration: 1m0.00080162s trace: https://jaeger.dev.storj.io/trace/1e577f05655b28db FAIL (2025-12-08 22:12:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000827318s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [......X...X.....XXX............................X.X..........] duration: 3.437368496s trace: https://jaeger.dev.storj.io/trace/6f4f8cb243e59639 FAIL (2025-12-08 22:12:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.437395187s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................X...............X............] duration: 5.891816546s trace: https://jaeger.dev.storj.io/trace/72871f7c52601eaf FAIL (2025-12-08 22:12:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== _pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917385480 (78h34m23.788468516s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917994611 (78h34m23.789077666s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918445960 (78h34m23.789528995s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918888933 (78h34m23.789971978s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919333861 (78h34m23.790416906s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919736458 (78h34m23.790819503s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920148312 (78h34m23.791231347s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920546540 (78h34m23.791629585s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.921942007 (78h34m23.793025052s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.923813691 (78h34m23.794896736s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.925353112 (78h34m23.796436157s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.927009226 (78h34m23.798092261s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.928697040 (78h34m23.799780085s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.931092261 (78h34m23.802175306s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.932748024 (78h34m23.803831069s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.933733821 (78h34m23.804816856s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.935468164 (78h34m23.806551209s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:25.369562765 (78h34m25.24064581s) [upload=68635]: Next returned nil segment Dec 8 22:03:25.369578857 (78h34m25.240661892s) [upload=68635]: Waiting for error group managing segments... Dec 8 22:03:25.775620953 (78h34m25.646703999s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.798803277 (78h34m25.669886322s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.801642202 (78h34m25.672725237s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.802057232 (78h34m25.673140267s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816132514 (78h34m25.687215559s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816446331 (78h34m25.687529376s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.869308801 (78h34m25.740391846s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.873476507 (78h34m25.744559542s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.907465318 (78h34m25.778548354s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.926737626 (78h34m25.797820671s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.928011191 (78h34m25.799094226s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.930540317 (78h34m25.801623362s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.954520891 (78h34m25.825603926s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001264168 (78h34m25.872347203s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001283154 (78h34m25.872366189s) [seg_pos={0 0}, upload=68635]: Segment reached optimal threshold of 14 pieces. Dec 8 22:03:26.001442167 (78h34m25.872525202s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001448459 (78h34m25.872531494s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001703023 (78h34m25.872786068s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123uv49k8LraXJfYQg6KRQthrYs1Egc4c3eZWm5JPvDatFbykm8): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001710758 (78h34m25.872793803s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001867176 (78h34m25.872950231s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Myourwn6UnUxVkxzBdRJApeAGNrZzjrgLgzAfki6x9JZ1Fy17): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001872526 (78h34m25.872955581s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002060293 (78h34m25.873143328s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHCVWGoutBZ3Du4xTCCJGALvrAZwB7cmC1vsRGj5LcwyfW8Qe): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.002066315 (78h34m25.873149351s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002085732 (78h34m25.873168767s) [seg_pos={0 0}, upload=68635]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 22:03:26.002156887 (78h34m25.873239922s) [upload=68635]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 22:03:26.002161526 (78h34m25.873244561s) [upload=68635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 22:04:50.380087807 (78h35m50.251170842s) [upload=68635]: Done uploading segments... Dec 8 22:04:50.380137331 (78h35m50.251220376s) [upload=68635]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.891852934s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................X...............X...........] duration: 5.766191259s trace: https://jaeger.dev.storj.io/trace/72871f7c52601eaf FAIL (2025-12-08 22:12:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== _pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917385480 (78h34m23.788468516s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917994611 (78h34m23.789077666s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918445960 (78h34m23.789528995s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918888933 (78h34m23.789971978s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919333861 (78h34m23.790416906s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919736458 (78h34m23.790819503s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920148312 (78h34m23.791231347s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920546540 (78h34m23.791629585s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.921942007 (78h34m23.793025052s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.923813691 (78h34m23.794896736s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.925353112 (78h34m23.796436157s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.927009226 (78h34m23.798092261s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.928697040 (78h34m23.799780085s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.931092261 (78h34m23.802175306s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.932748024 (78h34m23.803831069s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.933733821 (78h34m23.804816856s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.935468164 (78h34m23.806551209s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:25.369562765 (78h34m25.24064581s) [upload=68635]: Next returned nil segment Dec 8 22:03:25.369578857 (78h34m25.240661892s) [upload=68635]: Waiting for error group managing segments... Dec 8 22:03:25.775620953 (78h34m25.646703999s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.798803277 (78h34m25.669886322s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.801642202 (78h34m25.672725237s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.802057232 (78h34m25.673140267s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816132514 (78h34m25.687215559s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816446331 (78h34m25.687529376s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.869308801 (78h34m25.740391846s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.873476507 (78h34m25.744559542s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.907465318 (78h34m25.778548354s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.926737626 (78h34m25.797820671s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.928011191 (78h34m25.799094226s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.930540317 (78h34m25.801623362s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.954520891 (78h34m25.825603926s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001264168 (78h34m25.872347203s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001283154 (78h34m25.872366189s) [seg_pos={0 0}, upload=68635]: Segment reached optimal threshold of 14 pieces. Dec 8 22:03:26.001442167 (78h34m25.872525202s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001448459 (78h34m25.872531494s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001703023 (78h34m25.872786068s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123uv49k8LraXJfYQg6KRQthrYs1Egc4c3eZWm5JPvDatFbykm8): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001710758 (78h34m25.872793803s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001867176 (78h34m25.872950231s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Myourwn6UnUxVkxzBdRJApeAGNrZzjrgLgzAfki6x9JZ1Fy17): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001872526 (78h34m25.872955581s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002060293 (78h34m25.873143328s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHCVWGoutBZ3Du4xTCCJGALvrAZwB7cmC1vsRGj5LcwyfW8Qe): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.002066315 (78h34m25.873149351s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002085732 (78h34m25.873168767s) [seg_pos={0 0}, upload=68635]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 22:03:26.002156887 (78h34m25.873239922s) [upload=68635]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 22:03:26.002161526 (78h34m25.873244561s) [upload=68635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 22:04:50.380087807 (78h35m50.251170842s) [upload=68635]: Done uploading segments... Dec 8 22:04:50.380137331 (78h35m50.251220376s) [upload=68635]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.766223049s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 4.91578381s trace: https://jaeger.dev.storj.io/trace/51c602d400b15cea FAIL (2025-12-08 22:12:09 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.915813668s core uplink eu1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.765780254s trace: https://jaeger.dev.storj.io/trace/1aa6e2916420d374 FAIL (2025-12-08 22:12:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.765807335s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.064887127s trace: https://jaeger.dev.storj.io/trace/51c602d400b15cea FAIL (2025-12-08 22:11:52 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.064919119s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X...............X..........] duration: 28.763234463s trace: https://jaeger.dev.storj.io/trace/72871f7c52601eaf FAIL (2025-12-08 22:11:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== _pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917385480 (78h34m23.788468516s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917994611 (78h34m23.789077666s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918445960 (78h34m23.789528995s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918888933 (78h34m23.789971978s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919333861 (78h34m23.790416906s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919736458 (78h34m23.790819503s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920148312 (78h34m23.791231347s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920546540 (78h34m23.791629585s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.921942007 (78h34m23.793025052s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.923813691 (78h34m23.794896736s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.925353112 (78h34m23.796436157s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.927009226 (78h34m23.798092261s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.928697040 (78h34m23.799780085s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.931092261 (78h34m23.802175306s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.932748024 (78h34m23.803831069s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.933733821 (78h34m23.804816856s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.935468164 (78h34m23.806551209s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:25.369562765 (78h34m25.24064581s) [upload=68635]: Next returned nil segment Dec 8 22:03:25.369578857 (78h34m25.240661892s) [upload=68635]: Waiting for error group managing segments... Dec 8 22:03:25.775620953 (78h34m25.646703999s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.798803277 (78h34m25.669886322s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.801642202 (78h34m25.672725237s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.802057232 (78h34m25.673140267s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816132514 (78h34m25.687215559s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816446331 (78h34m25.687529376s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.869308801 (78h34m25.740391846s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.873476507 (78h34m25.744559542s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.907465318 (78h34m25.778548354s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.926737626 (78h34m25.797820671s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.928011191 (78h34m25.799094226s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.930540317 (78h34m25.801623362s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.954520891 (78h34m25.825603926s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001264168 (78h34m25.872347203s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001283154 (78h34m25.872366189s) [seg_pos={0 0}, upload=68635]: Segment reached optimal threshold of 14 pieces. Dec 8 22:03:26.001442167 (78h34m25.872525202s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001448459 (78h34m25.872531494s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001703023 (78h34m25.872786068s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123uv49k8LraXJfYQg6KRQthrYs1Egc4c3eZWm5JPvDatFbykm8): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001710758 (78h34m25.872793803s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001867176 (78h34m25.872950231s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Myourwn6UnUxVkxzBdRJApeAGNrZzjrgLgzAfki6x9JZ1Fy17): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001872526 (78h34m25.872955581s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002060293 (78h34m25.873143328s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHCVWGoutBZ3Du4xTCCJGALvrAZwB7cmC1vsRGj5LcwyfW8Qe): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.002066315 (78h34m25.873149351s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002085732 (78h34m25.873168767s) [seg_pos={0 0}, upload=68635]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 22:03:26.002156887 (78h34m25.873239922s) [upload=68635]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 22:03:26.002161526 (78h34m25.873244561s) [upload=68635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 22:04:50.380087807 (78h35m50.251170842s) [upload=68635]: Done uploading segments... Dec 8 22:04:50.380137331 (78h35m50.251220376s) [upload=68635]: Upload finished. err: metaclient: context deadline exceeded Duration: 28.763261894s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.211689636s trace: https://jaeger.dev.storj.io/trace/1aa6e2916420d374 FAIL (2025-12-08 22:11:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.211731355s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 2.040909205s trace: https://jaeger.dev.storj.io/trace/1ea7632defe62c27 FAIL (2025-12-08 22:11:35 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 2.04093777s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [..X..X....X.X..................................X..X.........] duration: 5.887094218s trace: https://jaeger.dev.storj.io/trace/6579bccb126e9260 FAIL (2025-12-08 22:11:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.887120678s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [.......X...X.....XXX............................X.X.........] duration: 3.521644178s trace: https://jaeger.dev.storj.io/trace/6f4f8cb243e59639 FAIL (2025-12-08 22:11:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.521664357s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.793246384s trace: https://jaeger.dev.storj.io/trace/1aa6e2916420d374 FAIL (2025-12-08 22:10:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.79327541s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [...X..X....X.X..................................X..X........] duration: 6.512416602s trace: https://jaeger.dev.storj.io/trace/6579bccb126e9260 FAIL (2025-12-08 22:10:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.512445568s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.055346983s trace: https://jaeger.dev.storj.io/trace/51c602d400b15cea FAIL (2025-12-08 22:10:43 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.055361141s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 1.98251862s trace: https://jaeger.dev.storj.io/trace/1ea7632defe62c27 FAIL (2025-12-08 22:10:30 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.982550501s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................X...............X.........] duration: 5.756491766s trace: https://jaeger.dev.storj.io/trace/72871f7c52601eaf FAIL (2025-12-08 22:10:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== _pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917385480 (78h34m23.788468516s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917994611 (78h34m23.789077666s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918445960 (78h34m23.789528995s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918888933 (78h34m23.789971978s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919333861 (78h34m23.790416906s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919736458 (78h34m23.790819503s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920148312 (78h34m23.791231347s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920546540 (78h34m23.791629585s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.921942007 (78h34m23.793025052s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.923813691 (78h34m23.794896736s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.925353112 (78h34m23.796436157s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.927009226 (78h34m23.798092261s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.928697040 (78h34m23.799780085s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.931092261 (78h34m23.802175306s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.932748024 (78h34m23.803831069s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.933733821 (78h34m23.804816856s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.935468164 (78h34m23.806551209s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:25.369562765 (78h34m25.24064581s) [upload=68635]: Next returned nil segment Dec 8 22:03:25.369578857 (78h34m25.240661892s) [upload=68635]: Waiting for error group managing segments... Dec 8 22:03:25.775620953 (78h34m25.646703999s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.798803277 (78h34m25.669886322s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.801642202 (78h34m25.672725237s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.802057232 (78h34m25.673140267s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816132514 (78h34m25.687215559s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816446331 (78h34m25.687529376s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.869308801 (78h34m25.740391846s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.873476507 (78h34m25.744559542s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.907465318 (78h34m25.778548354s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.926737626 (78h34m25.797820671s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.928011191 (78h34m25.799094226s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.930540317 (78h34m25.801623362s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.954520891 (78h34m25.825603926s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001264168 (78h34m25.872347203s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001283154 (78h34m25.872366189s) [seg_pos={0 0}, upload=68635]: Segment reached optimal threshold of 14 pieces. Dec 8 22:03:26.001442167 (78h34m25.872525202s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001448459 (78h34m25.872531494s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001703023 (78h34m25.872786068s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123uv49k8LraXJfYQg6KRQthrYs1Egc4c3eZWm5JPvDatFbykm8): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001710758 (78h34m25.872793803s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001867176 (78h34m25.872950231s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Myourwn6UnUxVkxzBdRJApeAGNrZzjrgLgzAfki6x9JZ1Fy17): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001872526 (78h34m25.872955581s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002060293 (78h34m25.873143328s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHCVWGoutBZ3Du4xTCCJGALvrAZwB7cmC1vsRGj5LcwyfW8Qe): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.002066315 (78h34m25.873149351s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002085732 (78h34m25.873168767s) [seg_pos={0 0}, upload=68635]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 22:03:26.002156887 (78h34m25.873239922s) [upload=68635]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 22:03:26.002161526 (78h34m25.873244561s) [upload=68635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 22:04:50.380087807 (78h35m50.251170842s) [upload=68635]: Done uploading segments... Dec 8 22:04:50.380137331 (78h35m50.251220376s) [upload=68635]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.75652557s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.426093469s trace: https://jaeger.dev.storj.io/trace/51c602d400b15cea FAIL (2025-12-08 22:10:18 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.426132505s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [........X...X.....XXX............................X.X........] duration: 3.39883619s trace: https://jaeger.dev.storj.io/trace/6f4f8cb243e59639 FAIL (2025-12-08 22:10:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.398884963s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................X...............X........] duration: 5.971015668s trace: https://jaeger.dev.storj.io/trace/72871f7c52601eaf FAIL (2025-12-08 22:10:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== _pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917385480 (78h34m23.788468516s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917994611 (78h34m23.789077666s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918445960 (78h34m23.789528995s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918888933 (78h34m23.789971978s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919333861 (78h34m23.790416906s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919736458 (78h34m23.790819503s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920148312 (78h34m23.791231347s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920546540 (78h34m23.791629585s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.921942007 (78h34m23.793025052s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.923813691 (78h34m23.794896736s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.925353112 (78h34m23.796436157s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.927009226 (78h34m23.798092261s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.928697040 (78h34m23.799780085s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.931092261 (78h34m23.802175306s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.932748024 (78h34m23.803831069s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.933733821 (78h34m23.804816856s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.935468164 (78h34m23.806551209s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:25.369562765 (78h34m25.24064581s) [upload=68635]: Next returned nil segment Dec 8 22:03:25.369578857 (78h34m25.240661892s) [upload=68635]: Waiting for error group managing segments... Dec 8 22:03:25.775620953 (78h34m25.646703999s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.798803277 (78h34m25.669886322s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.801642202 (78h34m25.672725237s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.802057232 (78h34m25.673140267s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816132514 (78h34m25.687215559s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816446331 (78h34m25.687529376s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.869308801 (78h34m25.740391846s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.873476507 (78h34m25.744559542s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.907465318 (78h34m25.778548354s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.926737626 (78h34m25.797820671s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.928011191 (78h34m25.799094226s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.930540317 (78h34m25.801623362s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.954520891 (78h34m25.825603926s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001264168 (78h34m25.872347203s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001283154 (78h34m25.872366189s) [seg_pos={0 0}, upload=68635]: Segment reached optimal threshold of 14 pieces. Dec 8 22:03:26.001442167 (78h34m25.872525202s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001448459 (78h34m25.872531494s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001703023 (78h34m25.872786068s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123uv49k8LraXJfYQg6KRQthrYs1Egc4c3eZWm5JPvDatFbykm8): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001710758 (78h34m25.872793803s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001867176 (78h34m25.872950231s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Myourwn6UnUxVkxzBdRJApeAGNrZzjrgLgzAfki6x9JZ1Fy17): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001872526 (78h34m25.872955581s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002060293 (78h34m25.873143328s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHCVWGoutBZ3Du4xTCCJGALvrAZwB7cmC1vsRGj5LcwyfW8Qe): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.002066315 (78h34m25.873149351s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002085732 (78h34m25.873168767s) [seg_pos={0 0}, upload=68635]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 22:03:26.002156887 (78h34m25.873239922s) [upload=68635]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 22:03:26.002161526 (78h34m25.873244561s) [upload=68635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 22:04:50.380087807 (78h35m50.251170842s) [upload=68635]: Done uploading segments... Dec 8 22:04:50.380137331 (78h35m50.251220376s) [upload=68635]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.971196334s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [....X..X....X.X..................................X..X.......] duration: 5.967669127s trace: https://jaeger.dev.storj.io/trace/6579bccb126e9260 FAIL (2025-12-08 22:09:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.96768648s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.86393671s trace: https://jaeger.dev.storj.io/trace/1aa6e2916420d374 FAIL (2025-12-08 22:09:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.863969623s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [.........X...X.....XXX............................X.X.......] duration: 3.48203659s trace: https://jaeger.dev.storj.io/trace/6f4f8cb243e59639 FAIL (2025-12-08 22:09:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.482063681s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.975588984s trace: https://jaeger.dev.storj.io/trace/1ea7632defe62c27 FAIL (2025-12-08 22:09:32 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 1.975618469s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 4.911445199s trace: https://jaeger.dev.storj.io/trace/51c602d400b15cea FAIL (2025-12-08 22:09:17 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.911477308s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.....X..X....X.X..................................X..X......] duration: 5.978775746s trace: https://jaeger.dev.storj.io/trace/6579bccb126e9260 FAIL (2025-12-08 22:09:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.978886397s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X...............X.......] duration: 5.870701604s trace: https://jaeger.dev.storj.io/trace/72871f7c52601eaf FAIL (2025-12-08 22:09:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== _pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917385480 (78h34m23.788468516s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917994611 (78h34m23.789077666s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918445960 (78h34m23.789528995s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918888933 (78h34m23.789971978s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919333861 (78h34m23.790416906s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919736458 (78h34m23.790819503s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920148312 (78h34m23.791231347s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920546540 (78h34m23.791629585s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.921942007 (78h34m23.793025052s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.923813691 (78h34m23.794896736s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.925353112 (78h34m23.796436157s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.927009226 (78h34m23.798092261s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.928697040 (78h34m23.799780085s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.931092261 (78h34m23.802175306s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.932748024 (78h34m23.803831069s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.933733821 (78h34m23.804816856s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.935468164 (78h34m23.806551209s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:25.369562765 (78h34m25.24064581s) [upload=68635]: Next returned nil segment Dec 8 22:03:25.369578857 (78h34m25.240661892s) [upload=68635]: Waiting for error group managing segments... Dec 8 22:03:25.775620953 (78h34m25.646703999s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.798803277 (78h34m25.669886322s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.801642202 (78h34m25.672725237s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.802057232 (78h34m25.673140267s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816132514 (78h34m25.687215559s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816446331 (78h34m25.687529376s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.869308801 (78h34m25.740391846s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.873476507 (78h34m25.744559542s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.907465318 (78h34m25.778548354s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.926737626 (78h34m25.797820671s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.928011191 (78h34m25.799094226s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.930540317 (78h34m25.801623362s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.954520891 (78h34m25.825603926s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001264168 (78h34m25.872347203s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001283154 (78h34m25.872366189s) [seg_pos={0 0}, upload=68635]: Segment reached optimal threshold of 14 pieces. Dec 8 22:03:26.001442167 (78h34m25.872525202s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001448459 (78h34m25.872531494s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001703023 (78h34m25.872786068s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123uv49k8LraXJfYQg6KRQthrYs1Egc4c3eZWm5JPvDatFbykm8): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001710758 (78h34m25.872793803s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001867176 (78h34m25.872950231s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Myourwn6UnUxVkxzBdRJApeAGNrZzjrgLgzAfki6x9JZ1Fy17): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001872526 (78h34m25.872955581s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002060293 (78h34m25.873143328s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHCVWGoutBZ3Du4xTCCJGALvrAZwB7cmC1vsRGj5LcwyfW8Qe): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.002066315 (78h34m25.873149351s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002085732 (78h34m25.873168767s) [seg_pos={0 0}, upload=68635]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 22:03:26.002156887 (78h34m25.873239922s) [upload=68635]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 22:03:26.002161526 (78h34m25.873244561s) [upload=68635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 22:04:50.380087807 (78h35m50.251170842s) [upload=68635]: Done uploading segments... Dec 8 22:04:50.380137331 (78h35m50.251220376s) [upload=68635]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.870730829s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.036118558s trace: https://jaeger.dev.storj.io/trace/1aa6e2916420d374 FAIL (2025-12-08 22:08:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.036175046s edge gateway us-select-1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.238961528s trace: https://jaeger.dev.storj.io/trace/51c602d400b15cea FAIL (2025-12-08 22:08:54 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.238993128s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 7 out of 60 is failed (threshold=6) [..........X...X.....XXX............................X.X......] duration: 3.472068778s trace: https://jaeger.dev.storj.io/trace/6f4f8cb243e59639 FAIL (2025-12-08 22:08:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.47210657s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X...............X......] duration: 5.746570911s trace: https://jaeger.dev.storj.io/trace/72871f7c52601eaf FAIL (2025-12-08 22:08:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== _pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917385480 (78h34m23.788468516s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.917994611 (78h34m23.789077666s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918445960 (78h34m23.789528995s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.918888933 (78h34m23.789971978s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919333861 (78h34m23.790416906s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.919736458 (78h34m23.790819503s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920148312 (78h34m23.791231347s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.920546540 (78h34m23.791629585s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.921942007 (78h34m23.793025052s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.923813691 (78h34m23.794896736s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.925353112 (78h34m23.796436157s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.927009226 (78h34m23.798092261s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.928697040 (78h34m23.799780085s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.931092261 (78h34m23.802175306s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.932748024 (78h34m23.803831069s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.933733821 (78h34m23.804816856s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:23.935468164 (78h34m23.806551209s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Uploading piece... Dec 8 22:03:25.369562765 (78h34m25.24064581s) [upload=68635]: Next returned nil segment Dec 8 22:03:25.369578857 (78h34m25.240661892s) [upload=68635]: Waiting for error group managing segments... Dec 8 22:03:25.775620953 (78h34m25.646703999s) [noise=true , address=64.49.13.26:4951 , piece_id=XKY7QGIG22ASYFNIS6FKWIAYPPKI6GHEACACEIUH6ZKBWTUXMKGQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.798803277 (78h34m25.669886322s) [noise=true , address=145.79.227.7:4307 , piece_id=NPJXZLKRP2DF5DQJVWVVQHHM6MMCANC2V7DKFFPPKJTUMUDTYD6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.801642202 (78h34m25.672725237s) [noise=true , address=145.79.227.33:4876 , piece_id=YFHARIAGOM34XJBLR6DMZ7XX3HZRM5TO3G33KNPMUKZWD4NNTCPA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.802057232 (78h34m25.673140267s) [noise=true , address=145.79.227.13:4831 , piece_id=PA6DXWTKRG5VTOXGZUMXGH4OZ5G33F4OVWYBYYCAJ65NHVTYGQAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816132514 (78h34m25.687215559s) [noise=true , address=193.149.164.143:4694 , piece_id=YB5D6BHSN5N7JFMDC4MOSSYYMVN54LUY6UYF5F3XTUTM6ZBFALCA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.816446331 (78h34m25.687529376s) [noise=true , address=145.79.227.21:4499 , piece_id=ERNE6RSZXWK662FT7VJPQF4SAW24T26HUMO53TOWXOFBYEQZ6J2A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.869308801 (78h34m25.740391846s) [noise=true , address=77.90.31.16:4767 , piece_id=KUBH54G6LYSDL3SY4PHCOT6PAS4BGTF3E77R3ECWJ7TJKWOSQYRA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.873476507 (78h34m25.744559542s) [noise=true , address=5.231.73.4:4006 , piece_id=63SLI6FBO3B3YME3HX4SY2ESSK7LA5PABUL2CEQIUCSTALEVBYDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.907465318 (78h34m25.778548354s) [noise=true , address=77.90.31.14:4677 , piece_id=MZ2RMJMOR45SHFXGGGU22YEQ35BDS452LNCASZ7YB3OCITH6FIDA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.926737626 (78h34m25.797820671s) [noise=true , address=77.90.31.21:4884 , piece_id=TDUSME4Y2CEQJCXP5LKRMS7IZLSVLXRJ32PCA5BOQNHXUA5YBO7A, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.928011191 (78h34m25.799094226s) [noise=true , address=45.91.116.12:4258 , piece_id=DHW72CRHKVV7ZETIQZ4D2TJ66SXMY6QTW67GK7M3C27O3FY2VZKA, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.930540317 (78h34m25.801623362s) [noise=true , address=77.90.31.18:4820 , piece_id=Q5WI2HI5B77A7WS335YDDME3UJFDF6PDGFUIXPVHXFU4RHJT3SZQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:25.954520891 (78h34m25.825603926s) [noise=true , address=77.90.31.17:4801 , piece_id=CIWMVWXXIIMM2K7Q6XU24FCNBUT46AQZ4FUFUCEROI5AJMB5CKAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001264168 (78h34m25.872347203s) [noise=true , address=77.90.31.10:4526 , piece_id=376JCI6NHB3K3T4S7T5OBQL2CPA6CZVUJEDF7HIKZU737MEDQTAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: <nil> Dec 8 22:03:26.001283154 (78h34m25.872366189s) [seg_pos={0 0}, upload=68635]: Segment reached optimal threshold of 14 pieces. Dec 8 22:03:26.001442167 (78h34m25.872525202s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kGo5nNz8GyroAGoMQ75xDo4pppWPVsW9f8trLCWH6CqVXWsmK): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001448459 (78h34m25.872531494s) [noise=true , address=145.79.227.40:4073 , piece_id=PKQLWWXB4PGEJNWBVRODLGTFTKGVDNBWTOMWIWTYDUGADWPD5WAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001703023 (78h34m25.872786068s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123uv49k8LraXJfYQg6KRQthrYs1Egc4c3eZWm5JPvDatFbykm8): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001710758 (78h34m25.872793803s) [noise=true , address=45.91.116.8:4066 , piece_id=GCUGVVC7DLF24CROYKYT4SYEAOFQ7L4RGIDLJH3JE2EO2FAISBYQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.001867176 (78h34m25.872950231s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Myourwn6UnUxVkxzBdRJApeAGNrZzjrgLgzAfki6x9JZ1Fy17): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.001872526 (78h34m25.872955581s) [noise=true , address=77.90.31.13:4637 , piece_id=U6YV74XHJ5Z47MR2XR3ZQY6DLFKYZ6PKESD4MRNUZ22UXNHIAX6Q, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002060293 (78h34m25.873143328s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHCVWGoutBZ3Du4xTCCJGALvrAZwB7cmC1vsRGj5LcwyfW8Qe): protocol: expected piece hash; context canceled; EOF Dec 8 22:03:26.002066315 (78h34m25.873149351s) [noise=true , address=45.91.116.4:4970 , piece_id=GRDPN2OXJLU6DMUPVOETE34QSVWTCAB5IFFO4MQRFSKUGMCVWPAQ, seg_pos={0 0}, upload=68635]: Long tail cancellation due to optimal threshold Dec 8 22:03:26.002085732 (78h34m25.873168767s) [seg_pos={0 0}, upload=68635]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 22:03:26.002156887 (78h34m25.873239922s) [upload=68635]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 22:03:26.002161526 (78h34m25.873244561s) [upload=68635]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 22:04:50.380087807 (78h35m50.251170842s) [upload=68635]: Done uploading segments... Dec 8 22:04:50.380137331 (78h35m50.251220376s) [upload=68635]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.746589717s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 7 out of 60 is failed (threshold=3) [X.....X..X....X.X..................................X..X.....] duration: 5.876174438s trace: https://jaeger.dev.storj.io/trace/6579bccb126e9260 FAIL (2025-12-08 22:08:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.876248779s core uplink us-select-1 us1-placement-20

1
...
11674
11675
11676
11677
11678
...
17318