Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................X...................................] duration: 3.644423702s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:16:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.644449042s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................X..................................] duration: 3.726598516s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:16:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.726667797s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................X.................................] duration: 3.483554755s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:14:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.48357822s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................X................................] duration: 3.624419499s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:14:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.624470296s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................X...............................] duration: 3.615479764s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:13:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.615573983s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................X..............................] duration: 3.643146614s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:12:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.643173787s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................X.............................] duration: 3.612039706s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:11:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.612064954s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................X............................] duration: 3.627510082s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:10:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.6275359s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................X...........................] duration: 3.520563787s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:10:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.520591319s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................X..........................] duration: 3.596755812s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:09:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.596798263s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................X.........................] duration: 3.599592531s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:08:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.599619975s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................X........................] duration: 3.569659076s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:08:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.569716434s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................X.......................] duration: 4.358107605s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:07:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.358133243s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................X......................] duration: 3.643027289s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:06:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.643058639s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................X.....................] duration: 3.620719913s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:06:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.620945862s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................X....................] duration: 3.683927142s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:05:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.68399987s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................X...................] duration: 3.537746115s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:04:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.537769882s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................X..................] duration: 3.453709763s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:04:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.453736884s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................X.................] duration: 3.604305016s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:03:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.604344882s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................X................] duration: 3.504071632s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:02:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.504144441s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................................X...............] duration: 3.564219514s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:01:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.564244362s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................................X..............] duration: 3.61623934s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:01:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.616262815s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.811762884s trace: https://jaeger.dev.storj.io/trace/5026b732c6f0fe22 FAIL (2025-12-22 11:01:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, Piece ID: 46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ) context deadline exceeded; piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ) context deadline exceeded ======LOGS====== 22 10:17:14.811047764 (402h48m14.682130809s) [upload=353639]: Starting upload Dec 22 10:17:14.811138286 (402h48m14.682221371s) [upload=353639]: Done starting upload Dec 22 10:17:14.811261631 (402h48m14.682344696s) [upload=353639]: Uploading segments... Dec 22 10:17:14.811582702 (402h48m14.682665747s) [upload=353639]: Got next segment. Inline: false Dec 22 10:17:14.811634691 (402h48m14.682717726s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 10:17:14.811670749 (402h48m14.682753784s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 10:17:15.062219720 (402h48m14.933302775s) [seg_pos={0 0}, upload=353639]: Begin upload segment... Dec 22 10:17:15.062379986 (402h48m14.933463021s) [seg_pos={0 0}, upload=353639]: Done begin upload segment. Dec 22 10:17:15.062398501 (402h48m14.933481536s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.062447905 (402h48m14.93353095s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.063165711 (402h48m14.934248767s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.064547112 (402h48m14.935630147s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065053717 (402h48m14.936136752s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065346284 (402h48m14.936429329s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065726608 (402h48m14.936809653s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066044513 (402h48m14.937127549s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066379291 (402h48m14.937462336s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066696966 (402h48m14.937780011s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067032044 (402h48m14.938115089s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067378333 (402h48m14.938461378s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.068500340 (402h48m14.939583375s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.069610214 (402h48m14.940693259s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.071423677 (402h48m14.942506722s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.072890550 (402h48m14.943973585s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:16.739208516 (402h48m16.610291551s) [upload=353639]: Next returned nil segment Dec 22 10:17:16.739225940 (402h48m16.610308985s) [upload=353639]: Waiting for error group managing segments... Dec 22 10:17:17.056843172 (402h48m16.927926217s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057237262 (402h48m16.928320307s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057619920 (402h48m16.928702965s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.071672710 (402h48m16.942755755s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.079155218 (402h48m16.950238263s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.080529355 (402h48m16.95161239s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.081308017 (402h48m16.952391062s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.087060581 (402h48m16.958143636s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.120430874 (402h48m16.991513909s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.131842193 (402h48m17.002925238s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.140731891 (402h48m17.011814936s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.325436603 (402h48m17.196519638s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339091925 (402h48m17.21017496s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339114328 (402h48m17.210197363s) [seg_pos={0 0}, upload=353639]: Segment reached optimal threshold of 13 pieces. Dec 22 10:17:17.339325330 (402h48m17.210408385s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339334477 (402h48m17.210417532s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339440859 (402h48m17.210523934s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339479453 (402h48m17.210562528s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339630210 (402h48m17.210713255s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection Dec 22 10:17:17.339642173 (402h48m17.210725208s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339663915 (402h48m17.21074695s) [seg_pos={0 0}, upload=353639]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 10:17:17.339746932 (402h48m17.210829967s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 10:17:17.339755538 (402h48m17.210838583s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 10:17:17.645053202 (402h48m17.516136237s) [upload=353639]: Done uploading segments... Dec 22 10:17:17.645073179 (402h48m17.516156234s) [upload=353639]: Upload finished. err: <nil> Duration: 5.811794283s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................................X.............] duration: 3.729158029s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 11:00:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.729254182s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 9.228876192s trace: https://jaeger.dev.storj.io/trace/5026b732c6f0fe22 FAIL (2025-12-22 11:00:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, Piece ID: 46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ) context deadline exceeded; piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ) context deadline exceeded ======LOGS====== 22 10:17:14.811047764 (402h48m14.682130809s) [upload=353639]: Starting upload Dec 22 10:17:14.811138286 (402h48m14.682221371s) [upload=353639]: Done starting upload Dec 22 10:17:14.811261631 (402h48m14.682344696s) [upload=353639]: Uploading segments... Dec 22 10:17:14.811582702 (402h48m14.682665747s) [upload=353639]: Got next segment. Inline: false Dec 22 10:17:14.811634691 (402h48m14.682717726s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 10:17:14.811670749 (402h48m14.682753784s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 10:17:15.062219720 (402h48m14.933302775s) [seg_pos={0 0}, upload=353639]: Begin upload segment... Dec 22 10:17:15.062379986 (402h48m14.933463021s) [seg_pos={0 0}, upload=353639]: Done begin upload segment. Dec 22 10:17:15.062398501 (402h48m14.933481536s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.062447905 (402h48m14.93353095s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.063165711 (402h48m14.934248767s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.064547112 (402h48m14.935630147s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065053717 (402h48m14.936136752s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065346284 (402h48m14.936429329s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065726608 (402h48m14.936809653s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066044513 (402h48m14.937127549s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066379291 (402h48m14.937462336s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066696966 (402h48m14.937780011s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067032044 (402h48m14.938115089s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067378333 (402h48m14.938461378s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.068500340 (402h48m14.939583375s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.069610214 (402h48m14.940693259s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.071423677 (402h48m14.942506722s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.072890550 (402h48m14.943973585s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:16.739208516 (402h48m16.610291551s) [upload=353639]: Next returned nil segment Dec 22 10:17:16.739225940 (402h48m16.610308985s) [upload=353639]: Waiting for error group managing segments... Dec 22 10:17:17.056843172 (402h48m16.927926217s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057237262 (402h48m16.928320307s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057619920 (402h48m16.928702965s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.071672710 (402h48m16.942755755s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.079155218 (402h48m16.950238263s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.080529355 (402h48m16.95161239s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.081308017 (402h48m16.952391062s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.087060581 (402h48m16.958143636s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.120430874 (402h48m16.991513909s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.131842193 (402h48m17.002925238s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.140731891 (402h48m17.011814936s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.325436603 (402h48m17.196519638s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339091925 (402h48m17.21017496s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339114328 (402h48m17.210197363s) [seg_pos={0 0}, upload=353639]: Segment reached optimal threshold of 13 pieces. Dec 22 10:17:17.339325330 (402h48m17.210408385s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339334477 (402h48m17.210417532s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339440859 (402h48m17.210523934s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339479453 (402h48m17.210562528s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339630210 (402h48m17.210713255s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection Dec 22 10:17:17.339642173 (402h48m17.210725208s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339663915 (402h48m17.21074695s) [seg_pos={0 0}, upload=353639]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 10:17:17.339746932 (402h48m17.210829967s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 10:17:17.339755538 (402h48m17.210838583s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 10:17:17.645053202 (402h48m17.516136237s) [upload=353639]: Done uploading segments... Dec 22 10:17:17.645073179 (402h48m17.516156234s) [upload=353639]: Upload finished. err: <nil> Duration: 9.228898777s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................................X............] duration: 3.610774376s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 10:59:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.610788944s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................................X...........] duration: 3.39451673s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 10:58:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.394571915s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.76995796s trace: https://jaeger.dev.storj.io/trace/5026b732c6f0fe22 FAIL (2025-12-22 10:58:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, Piece ID: 46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ) context deadline exceeded; piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ) context deadline exceeded ======LOGS====== 22 10:17:14.811047764 (402h48m14.682130809s) [upload=353639]: Starting upload Dec 22 10:17:14.811138286 (402h48m14.682221371s) [upload=353639]: Done starting upload Dec 22 10:17:14.811261631 (402h48m14.682344696s) [upload=353639]: Uploading segments... Dec 22 10:17:14.811582702 (402h48m14.682665747s) [upload=353639]: Got next segment. Inline: false Dec 22 10:17:14.811634691 (402h48m14.682717726s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 10:17:14.811670749 (402h48m14.682753784s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 10:17:15.062219720 (402h48m14.933302775s) [seg_pos={0 0}, upload=353639]: Begin upload segment... Dec 22 10:17:15.062379986 (402h48m14.933463021s) [seg_pos={0 0}, upload=353639]: Done begin upload segment. Dec 22 10:17:15.062398501 (402h48m14.933481536s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.062447905 (402h48m14.93353095s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.063165711 (402h48m14.934248767s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.064547112 (402h48m14.935630147s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065053717 (402h48m14.936136752s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065346284 (402h48m14.936429329s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065726608 (402h48m14.936809653s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066044513 (402h48m14.937127549s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066379291 (402h48m14.937462336s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066696966 (402h48m14.937780011s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067032044 (402h48m14.938115089s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067378333 (402h48m14.938461378s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.068500340 (402h48m14.939583375s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.069610214 (402h48m14.940693259s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.071423677 (402h48m14.942506722s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.072890550 (402h48m14.943973585s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:16.739208516 (402h48m16.610291551s) [upload=353639]: Next returned nil segment Dec 22 10:17:16.739225940 (402h48m16.610308985s) [upload=353639]: Waiting for error group managing segments... Dec 22 10:17:17.056843172 (402h48m16.927926217s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057237262 (402h48m16.928320307s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057619920 (402h48m16.928702965s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.071672710 (402h48m16.942755755s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.079155218 (402h48m16.950238263s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.080529355 (402h48m16.95161239s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.081308017 (402h48m16.952391062s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.087060581 (402h48m16.958143636s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.120430874 (402h48m16.991513909s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.131842193 (402h48m17.002925238s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.140731891 (402h48m17.011814936s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.325436603 (402h48m17.196519638s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339091925 (402h48m17.21017496s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339114328 (402h48m17.210197363s) [seg_pos={0 0}, upload=353639]: Segment reached optimal threshold of 13 pieces. Dec 22 10:17:17.339325330 (402h48m17.210408385s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339334477 (402h48m17.210417532s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339440859 (402h48m17.210523934s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339479453 (402h48m17.210562528s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339630210 (402h48m17.210713255s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection Dec 22 10:17:17.339642173 (402h48m17.210725208s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339663915 (402h48m17.21074695s) [seg_pos={0 0}, upload=353639]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 10:17:17.339746932 (402h48m17.210829967s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 10:17:17.339755538 (402h48m17.210838583s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 10:17:17.645053202 (402h48m17.516136237s) [upload=353639]: Done uploading segments... Dec 22 10:17:17.645073179 (402h48m17.516156234s) [upload=353639]: Upload finished. err: <nil> Duration: 5.769984381s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................................X..........] duration: 3.605402124s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 10:58:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.605430118s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.765688419s trace: https://jaeger.dev.storj.io/trace/5026b732c6f0fe22 FAIL (2025-12-22 10:58:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, Piece ID: 46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ) context deadline exceeded; piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ) context deadline exceeded ======LOGS====== 22 10:17:14.811047764 (402h48m14.682130809s) [upload=353639]: Starting upload Dec 22 10:17:14.811138286 (402h48m14.682221371s) [upload=353639]: Done starting upload Dec 22 10:17:14.811261631 (402h48m14.682344696s) [upload=353639]: Uploading segments... Dec 22 10:17:14.811582702 (402h48m14.682665747s) [upload=353639]: Got next segment. Inline: false Dec 22 10:17:14.811634691 (402h48m14.682717726s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 10:17:14.811670749 (402h48m14.682753784s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 10:17:15.062219720 (402h48m14.933302775s) [seg_pos={0 0}, upload=353639]: Begin upload segment... Dec 22 10:17:15.062379986 (402h48m14.933463021s) [seg_pos={0 0}, upload=353639]: Done begin upload segment. Dec 22 10:17:15.062398501 (402h48m14.933481536s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.062447905 (402h48m14.93353095s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.063165711 (402h48m14.934248767s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.064547112 (402h48m14.935630147s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065053717 (402h48m14.936136752s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065346284 (402h48m14.936429329s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065726608 (402h48m14.936809653s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066044513 (402h48m14.937127549s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066379291 (402h48m14.937462336s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066696966 (402h48m14.937780011s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067032044 (402h48m14.938115089s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067378333 (402h48m14.938461378s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.068500340 (402h48m14.939583375s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.069610214 (402h48m14.940693259s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.071423677 (402h48m14.942506722s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.072890550 (402h48m14.943973585s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:16.739208516 (402h48m16.610291551s) [upload=353639]: Next returned nil segment Dec 22 10:17:16.739225940 (402h48m16.610308985s) [upload=353639]: Waiting for error group managing segments... Dec 22 10:17:17.056843172 (402h48m16.927926217s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057237262 (402h48m16.928320307s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057619920 (402h48m16.928702965s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.071672710 (402h48m16.942755755s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.079155218 (402h48m16.950238263s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.080529355 (402h48m16.95161239s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.081308017 (402h48m16.952391062s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.087060581 (402h48m16.958143636s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.120430874 (402h48m16.991513909s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.131842193 (402h48m17.002925238s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.140731891 (402h48m17.011814936s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.325436603 (402h48m17.196519638s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339091925 (402h48m17.21017496s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339114328 (402h48m17.210197363s) [seg_pos={0 0}, upload=353639]: Segment reached optimal threshold of 13 pieces. Dec 22 10:17:17.339325330 (402h48m17.210408385s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339334477 (402h48m17.210417532s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339440859 (402h48m17.210523934s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339479453 (402h48m17.210562528s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339630210 (402h48m17.210713255s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection Dec 22 10:17:17.339642173 (402h48m17.210725208s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339663915 (402h48m17.21074695s) [seg_pos={0 0}, upload=353639]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 10:17:17.339746932 (402h48m17.210829967s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 10:17:17.339755538 (402h48m17.210838583s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 10:17:17.645053202 (402h48m17.516136237s) [upload=353639]: Done uploading segments... Dec 22 10:17:17.645073179 (402h48m17.516156234s) [upload=353639]: Upload finished. err: <nil> Duration: 5.765735248s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................................X.........] duration: 3.424354506s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 10:58:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.424379683s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................................X........] duration: 3.608966933s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 10:57:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.608994676s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.507641105s trace: https://jaeger.dev.storj.io/trace/5026b732c6f0fe22 FAIL (2025-12-22 10:57:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, Piece ID: 46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ) context deadline exceeded; piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ) context deadline exceeded ======LOGS====== 22 10:17:14.811047764 (402h48m14.682130809s) [upload=353639]: Starting upload Dec 22 10:17:14.811138286 (402h48m14.682221371s) [upload=353639]: Done starting upload Dec 22 10:17:14.811261631 (402h48m14.682344696s) [upload=353639]: Uploading segments... Dec 22 10:17:14.811582702 (402h48m14.682665747s) [upload=353639]: Got next segment. Inline: false Dec 22 10:17:14.811634691 (402h48m14.682717726s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 10:17:14.811670749 (402h48m14.682753784s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 10:17:15.062219720 (402h48m14.933302775s) [seg_pos={0 0}, upload=353639]: Begin upload segment... Dec 22 10:17:15.062379986 (402h48m14.933463021s) [seg_pos={0 0}, upload=353639]: Done begin upload segment. Dec 22 10:17:15.062398501 (402h48m14.933481536s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.062447905 (402h48m14.93353095s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.063165711 (402h48m14.934248767s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.064547112 (402h48m14.935630147s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065053717 (402h48m14.936136752s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065346284 (402h48m14.936429329s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065726608 (402h48m14.936809653s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066044513 (402h48m14.937127549s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066379291 (402h48m14.937462336s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066696966 (402h48m14.937780011s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067032044 (402h48m14.938115089s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067378333 (402h48m14.938461378s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.068500340 (402h48m14.939583375s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.069610214 (402h48m14.940693259s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.071423677 (402h48m14.942506722s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.072890550 (402h48m14.943973585s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:16.739208516 (402h48m16.610291551s) [upload=353639]: Next returned nil segment Dec 22 10:17:16.739225940 (402h48m16.610308985s) [upload=353639]: Waiting for error group managing segments... Dec 22 10:17:17.056843172 (402h48m16.927926217s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057237262 (402h48m16.928320307s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057619920 (402h48m16.928702965s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.071672710 (402h48m16.942755755s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.079155218 (402h48m16.950238263s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.080529355 (402h48m16.95161239s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.081308017 (402h48m16.952391062s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.087060581 (402h48m16.958143636s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.120430874 (402h48m16.991513909s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.131842193 (402h48m17.002925238s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.140731891 (402h48m17.011814936s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.325436603 (402h48m17.196519638s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339091925 (402h48m17.21017496s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339114328 (402h48m17.210197363s) [seg_pos={0 0}, upload=353639]: Segment reached optimal threshold of 13 pieces. Dec 22 10:17:17.339325330 (402h48m17.210408385s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339334477 (402h48m17.210417532s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339440859 (402h48m17.210523934s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339479453 (402h48m17.210562528s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339630210 (402h48m17.210713255s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection Dec 22 10:17:17.339642173 (402h48m17.210725208s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339663915 (402h48m17.21074695s) [seg_pos={0 0}, upload=353639]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 10:17:17.339746932 (402h48m17.210829967s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 10:17:17.339755538 (402h48m17.210838583s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 10:17:17.645053202 (402h48m17.516136237s) [upload=353639]: Done uploading segments... Dec 22 10:17:17.645073179 (402h48m17.516156234s) [upload=353639]: Upload finished. err: <nil> Duration: 6.507667795s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................................X.......] duration: 3.497902495s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 10:57:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.49792704s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.889061707s trace: https://jaeger.dev.storj.io/trace/5026b732c6f0fe22 FAIL (2025-12-22 10:56:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, Piece ID: 46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ) context deadline exceeded; piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ) context deadline exceeded ======LOGS====== 22 10:17:14.811047764 (402h48m14.682130809s) [upload=353639]: Starting upload Dec 22 10:17:14.811138286 (402h48m14.682221371s) [upload=353639]: Done starting upload Dec 22 10:17:14.811261631 (402h48m14.682344696s) [upload=353639]: Uploading segments... Dec 22 10:17:14.811582702 (402h48m14.682665747s) [upload=353639]: Got next segment. Inline: false Dec 22 10:17:14.811634691 (402h48m14.682717726s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 10:17:14.811670749 (402h48m14.682753784s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 10:17:15.062219720 (402h48m14.933302775s) [seg_pos={0 0}, upload=353639]: Begin upload segment... Dec 22 10:17:15.062379986 (402h48m14.933463021s) [seg_pos={0 0}, upload=353639]: Done begin upload segment. Dec 22 10:17:15.062398501 (402h48m14.933481536s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.062447905 (402h48m14.93353095s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.063165711 (402h48m14.934248767s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.064547112 (402h48m14.935630147s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065053717 (402h48m14.936136752s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065346284 (402h48m14.936429329s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065726608 (402h48m14.936809653s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066044513 (402h48m14.937127549s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066379291 (402h48m14.937462336s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066696966 (402h48m14.937780011s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067032044 (402h48m14.938115089s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067378333 (402h48m14.938461378s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.068500340 (402h48m14.939583375s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.069610214 (402h48m14.940693259s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.071423677 (402h48m14.942506722s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.072890550 (402h48m14.943973585s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:16.739208516 (402h48m16.610291551s) [upload=353639]: Next returned nil segment Dec 22 10:17:16.739225940 (402h48m16.610308985s) [upload=353639]: Waiting for error group managing segments... Dec 22 10:17:17.056843172 (402h48m16.927926217s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057237262 (402h48m16.928320307s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057619920 (402h48m16.928702965s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.071672710 (402h48m16.942755755s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.079155218 (402h48m16.950238263s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.080529355 (402h48m16.95161239s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.081308017 (402h48m16.952391062s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.087060581 (402h48m16.958143636s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.120430874 (402h48m16.991513909s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.131842193 (402h48m17.002925238s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.140731891 (402h48m17.011814936s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.325436603 (402h48m17.196519638s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339091925 (402h48m17.21017496s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339114328 (402h48m17.210197363s) [seg_pos={0 0}, upload=353639]: Segment reached optimal threshold of 13 pieces. Dec 22 10:17:17.339325330 (402h48m17.210408385s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339334477 (402h48m17.210417532s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339440859 (402h48m17.210523934s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339479453 (402h48m17.210562528s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339630210 (402h48m17.210713255s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection Dec 22 10:17:17.339642173 (402h48m17.210725208s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339663915 (402h48m17.21074695s) [seg_pos={0 0}, upload=353639]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 10:17:17.339746932 (402h48m17.210829967s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 10:17:17.339755538 (402h48m17.210838583s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 10:17:17.645053202 (402h48m17.516136237s) [upload=353639]: Done uploading segments... Dec 22 10:17:17.645073179 (402h48m17.516156234s) [upload=353639]: Upload finished. err: <nil> Duration: 5.889088138s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................................X......] duration: 3.721293502s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 10:56:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.721319643s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.846325737s trace: https://jaeger.dev.storj.io/trace/5026b732c6f0fe22 FAIL (2025-12-22 10:56:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, Piece ID: 46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ) context deadline exceeded; piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ) context deadline exceeded ======LOGS====== 22 10:17:14.811047764 (402h48m14.682130809s) [upload=353639]: Starting upload Dec 22 10:17:14.811138286 (402h48m14.682221371s) [upload=353639]: Done starting upload Dec 22 10:17:14.811261631 (402h48m14.682344696s) [upload=353639]: Uploading segments... Dec 22 10:17:14.811582702 (402h48m14.682665747s) [upload=353639]: Got next segment. Inline: false Dec 22 10:17:14.811634691 (402h48m14.682717726s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 10:17:14.811670749 (402h48m14.682753784s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 10:17:15.062219720 (402h48m14.933302775s) [seg_pos={0 0}, upload=353639]: Begin upload segment... Dec 22 10:17:15.062379986 (402h48m14.933463021s) [seg_pos={0 0}, upload=353639]: Done begin upload segment. Dec 22 10:17:15.062398501 (402h48m14.933481536s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.062447905 (402h48m14.93353095s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.063165711 (402h48m14.934248767s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.064547112 (402h48m14.935630147s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065053717 (402h48m14.936136752s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065346284 (402h48m14.936429329s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065726608 (402h48m14.936809653s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066044513 (402h48m14.937127549s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066379291 (402h48m14.937462336s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066696966 (402h48m14.937780011s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067032044 (402h48m14.938115089s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067378333 (402h48m14.938461378s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.068500340 (402h48m14.939583375s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.069610214 (402h48m14.940693259s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.071423677 (402h48m14.942506722s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.072890550 (402h48m14.943973585s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:16.739208516 (402h48m16.610291551s) [upload=353639]: Next returned nil segment Dec 22 10:17:16.739225940 (402h48m16.610308985s) [upload=353639]: Waiting for error group managing segments... Dec 22 10:17:17.056843172 (402h48m16.927926217s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057237262 (402h48m16.928320307s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057619920 (402h48m16.928702965s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.071672710 (402h48m16.942755755s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.079155218 (402h48m16.950238263s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.080529355 (402h48m16.95161239s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.081308017 (402h48m16.952391062s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.087060581 (402h48m16.958143636s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.120430874 (402h48m16.991513909s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.131842193 (402h48m17.002925238s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.140731891 (402h48m17.011814936s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.325436603 (402h48m17.196519638s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339091925 (402h48m17.21017496s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339114328 (402h48m17.210197363s) [seg_pos={0 0}, upload=353639]: Segment reached optimal threshold of 13 pieces. Dec 22 10:17:17.339325330 (402h48m17.210408385s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339334477 (402h48m17.210417532s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339440859 (402h48m17.210523934s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339479453 (402h48m17.210562528s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339630210 (402h48m17.210713255s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection Dec 22 10:17:17.339642173 (402h48m17.210725208s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339663915 (402h48m17.21074695s) [seg_pos={0 0}, upload=353639]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 10:17:17.339746932 (402h48m17.210829967s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 10:17:17.339755538 (402h48m17.210838583s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 10:17:17.645053202 (402h48m17.516136237s) [upload=353639]: Done uploading segments... Dec 22 10:17:17.645073179 (402h48m17.516156234s) [upload=353639]: Upload finished. err: <nil> Duration: 5.846353542s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.907652517s trace: https://jaeger.dev.storj.io/trace/5026b732c6f0fe22 FAIL (2025-12-22 10:56:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, Piece ID: 46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ) context deadline exceeded; piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ) context deadline exceeded ======LOGS====== 22 10:17:14.811047764 (402h48m14.682130809s) [upload=353639]: Starting upload Dec 22 10:17:14.811138286 (402h48m14.682221371s) [upload=353639]: Done starting upload Dec 22 10:17:14.811261631 (402h48m14.682344696s) [upload=353639]: Uploading segments... Dec 22 10:17:14.811582702 (402h48m14.682665747s) [upload=353639]: Got next segment. Inline: false Dec 22 10:17:14.811634691 (402h48m14.682717726s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 10:17:14.811670749 (402h48m14.682753784s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 10:17:15.062219720 (402h48m14.933302775s) [seg_pos={0 0}, upload=353639]: Begin upload segment... Dec 22 10:17:15.062379986 (402h48m14.933463021s) [seg_pos={0 0}, upload=353639]: Done begin upload segment. Dec 22 10:17:15.062398501 (402h48m14.933481536s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.062447905 (402h48m14.93353095s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.063165711 (402h48m14.934248767s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.064547112 (402h48m14.935630147s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065053717 (402h48m14.936136752s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065346284 (402h48m14.936429329s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065726608 (402h48m14.936809653s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066044513 (402h48m14.937127549s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066379291 (402h48m14.937462336s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066696966 (402h48m14.937780011s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067032044 (402h48m14.938115089s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067378333 (402h48m14.938461378s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.068500340 (402h48m14.939583375s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.069610214 (402h48m14.940693259s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.071423677 (402h48m14.942506722s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.072890550 (402h48m14.943973585s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:16.739208516 (402h48m16.610291551s) [upload=353639]: Next returned nil segment Dec 22 10:17:16.739225940 (402h48m16.610308985s) [upload=353639]: Waiting for error group managing segments... Dec 22 10:17:17.056843172 (402h48m16.927926217s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057237262 (402h48m16.928320307s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057619920 (402h48m16.928702965s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.071672710 (402h48m16.942755755s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.079155218 (402h48m16.950238263s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.080529355 (402h48m16.95161239s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.081308017 (402h48m16.952391062s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.087060581 (402h48m16.958143636s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.120430874 (402h48m16.991513909s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.131842193 (402h48m17.002925238s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.140731891 (402h48m17.011814936s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.325436603 (402h48m17.196519638s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339091925 (402h48m17.21017496s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339114328 (402h48m17.210197363s) [seg_pos={0 0}, upload=353639]: Segment reached optimal threshold of 13 pieces. Dec 22 10:17:17.339325330 (402h48m17.210408385s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339334477 (402h48m17.210417532s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339440859 (402h48m17.210523934s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339479453 (402h48m17.210562528s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339630210 (402h48m17.210713255s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection Dec 22 10:17:17.339642173 (402h48m17.210725208s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339663915 (402h48m17.21074695s) [seg_pos={0 0}, upload=353639]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 10:17:17.339746932 (402h48m17.210829967s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 10:17:17.339755538 (402h48m17.210838583s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 10:17:17.645053202 (402h48m17.516136237s) [upload=353639]: Done uploading segments... Dec 22 10:17:17.645073179 (402h48m17.516156234s) [upload=353639]: Upload finished. err: <nil> Duration: 5.907677946s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................................X.....] duration: 3.720916156s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 10:55:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.720942775s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.736705106s trace: https://jaeger.dev.storj.io/trace/5026b732c6f0fe22 FAIL (2025-12-22 10:55:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, Piece ID: 46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ) context deadline exceeded; piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ) context deadline exceeded ======LOGS====== 22 10:17:14.811047764 (402h48m14.682130809s) [upload=353639]: Starting upload Dec 22 10:17:14.811138286 (402h48m14.682221371s) [upload=353639]: Done starting upload Dec 22 10:17:14.811261631 (402h48m14.682344696s) [upload=353639]: Uploading segments... Dec 22 10:17:14.811582702 (402h48m14.682665747s) [upload=353639]: Got next segment. Inline: false Dec 22 10:17:14.811634691 (402h48m14.682717726s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 10:17:14.811670749 (402h48m14.682753784s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 10:17:15.062219720 (402h48m14.933302775s) [seg_pos={0 0}, upload=353639]: Begin upload segment... Dec 22 10:17:15.062379986 (402h48m14.933463021s) [seg_pos={0 0}, upload=353639]: Done begin upload segment. Dec 22 10:17:15.062398501 (402h48m14.933481536s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.062447905 (402h48m14.93353095s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.063165711 (402h48m14.934248767s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.064547112 (402h48m14.935630147s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065053717 (402h48m14.936136752s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065346284 (402h48m14.936429329s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065726608 (402h48m14.936809653s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066044513 (402h48m14.937127549s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066379291 (402h48m14.937462336s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066696966 (402h48m14.937780011s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067032044 (402h48m14.938115089s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067378333 (402h48m14.938461378s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.068500340 (402h48m14.939583375s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.069610214 (402h48m14.940693259s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.071423677 (402h48m14.942506722s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.072890550 (402h48m14.943973585s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:16.739208516 (402h48m16.610291551s) [upload=353639]: Next returned nil segment Dec 22 10:17:16.739225940 (402h48m16.610308985s) [upload=353639]: Waiting for error group managing segments... Dec 22 10:17:17.056843172 (402h48m16.927926217s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057237262 (402h48m16.928320307s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057619920 (402h48m16.928702965s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.071672710 (402h48m16.942755755s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.079155218 (402h48m16.950238263s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.080529355 (402h48m16.95161239s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.081308017 (402h48m16.952391062s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.087060581 (402h48m16.958143636s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.120430874 (402h48m16.991513909s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.131842193 (402h48m17.002925238s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.140731891 (402h48m17.011814936s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.325436603 (402h48m17.196519638s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339091925 (402h48m17.21017496s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339114328 (402h48m17.210197363s) [seg_pos={0 0}, upload=353639]: Segment reached optimal threshold of 13 pieces. Dec 22 10:17:17.339325330 (402h48m17.210408385s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339334477 (402h48m17.210417532s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339440859 (402h48m17.210523934s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339479453 (402h48m17.210562528s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339630210 (402h48m17.210713255s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection Dec 22 10:17:17.339642173 (402h48m17.210725208s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339663915 (402h48m17.21074695s) [seg_pos={0 0}, upload=353639]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 10:17:17.339746932 (402h48m17.210829967s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 10:17:17.339755538 (402h48m17.210838583s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 10:17:17.645053202 (402h48m17.516136237s) [upload=353639]: Done uploading segments... Dec 22 10:17:17.645073179 (402h48m17.516156234s) [upload=353639]: Upload finished. err: <nil> Duration: 5.736732136s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.772868728s trace: https://jaeger.dev.storj.io/trace/5026b732c6f0fe22 FAIL (2025-12-22 10:54:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, Piece ID: 46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ) context deadline exceeded; piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ) context deadline exceeded ======LOGS====== 22 10:17:14.811047764 (402h48m14.682130809s) [upload=353639]: Starting upload Dec 22 10:17:14.811138286 (402h48m14.682221371s) [upload=353639]: Done starting upload Dec 22 10:17:14.811261631 (402h48m14.682344696s) [upload=353639]: Uploading segments... Dec 22 10:17:14.811582702 (402h48m14.682665747s) [upload=353639]: Got next segment. Inline: false Dec 22 10:17:14.811634691 (402h48m14.682717726s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 10:17:14.811670749 (402h48m14.682753784s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 10:17:15.062219720 (402h48m14.933302775s) [seg_pos={0 0}, upload=353639]: Begin upload segment... Dec 22 10:17:15.062379986 (402h48m14.933463021s) [seg_pos={0 0}, upload=353639]: Done begin upload segment. Dec 22 10:17:15.062398501 (402h48m14.933481536s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.062447905 (402h48m14.93353095s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.063165711 (402h48m14.934248767s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.064547112 (402h48m14.935630147s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065053717 (402h48m14.936136752s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065346284 (402h48m14.936429329s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065726608 (402h48m14.936809653s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066044513 (402h48m14.937127549s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066379291 (402h48m14.937462336s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066696966 (402h48m14.937780011s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067032044 (402h48m14.938115089s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067378333 (402h48m14.938461378s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.068500340 (402h48m14.939583375s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.069610214 (402h48m14.940693259s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.071423677 (402h48m14.942506722s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.072890550 (402h48m14.943973585s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:16.739208516 (402h48m16.610291551s) [upload=353639]: Next returned nil segment Dec 22 10:17:16.739225940 (402h48m16.610308985s) [upload=353639]: Waiting for error group managing segments... Dec 22 10:17:17.056843172 (402h48m16.927926217s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057237262 (402h48m16.928320307s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057619920 (402h48m16.928702965s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.071672710 (402h48m16.942755755s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.079155218 (402h48m16.950238263s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.080529355 (402h48m16.95161239s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.081308017 (402h48m16.952391062s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.087060581 (402h48m16.958143636s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.120430874 (402h48m16.991513909s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.131842193 (402h48m17.002925238s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.140731891 (402h48m17.011814936s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.325436603 (402h48m17.196519638s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339091925 (402h48m17.21017496s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339114328 (402h48m17.210197363s) [seg_pos={0 0}, upload=353639]: Segment reached optimal threshold of 13 pieces. Dec 22 10:17:17.339325330 (402h48m17.210408385s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339334477 (402h48m17.210417532s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339440859 (402h48m17.210523934s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339479453 (402h48m17.210562528s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339630210 (402h48m17.210713255s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection Dec 22 10:17:17.339642173 (402h48m17.210725208s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339663915 (402h48m17.21074695s) [seg_pos={0 0}, upload=353639]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 10:17:17.339746932 (402h48m17.210829967s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 10:17:17.339755538 (402h48m17.210838583s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 10:17:17.645053202 (402h48m17.516136237s) [upload=353639]: Done uploading segments... Dec 22 10:17:17.645073179 (402h48m17.516156234s) [upload=353639]: Upload finished. err: <nil> Duration: 5.772900779s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 3.625616068s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 10:54:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.625640213s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.713110567s trace: https://jaeger.dev.storj.io/trace/5026b732c6f0fe22 FAIL (2025-12-22 10:54:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, Piece ID: 46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ) context deadline exceeded; piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ) context deadline exceeded ======LOGS====== 22 10:17:14.811047764 (402h48m14.682130809s) [upload=353639]: Starting upload Dec 22 10:17:14.811138286 (402h48m14.682221371s) [upload=353639]: Done starting upload Dec 22 10:17:14.811261631 (402h48m14.682344696s) [upload=353639]: Uploading segments... Dec 22 10:17:14.811582702 (402h48m14.682665747s) [upload=353639]: Got next segment. Inline: false Dec 22 10:17:14.811634691 (402h48m14.682717726s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 10:17:14.811670749 (402h48m14.682753784s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 10:17:15.062219720 (402h48m14.933302775s) [seg_pos={0 0}, upload=353639]: Begin upload segment... Dec 22 10:17:15.062379986 (402h48m14.933463021s) [seg_pos={0 0}, upload=353639]: Done begin upload segment. Dec 22 10:17:15.062398501 (402h48m14.933481536s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.062447905 (402h48m14.93353095s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.063165711 (402h48m14.934248767s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.064547112 (402h48m14.935630147s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065053717 (402h48m14.936136752s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065346284 (402h48m14.936429329s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065726608 (402h48m14.936809653s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066044513 (402h48m14.937127549s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066379291 (402h48m14.937462336s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066696966 (402h48m14.937780011s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067032044 (402h48m14.938115089s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067378333 (402h48m14.938461378s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.068500340 (402h48m14.939583375s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.069610214 (402h48m14.940693259s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.071423677 (402h48m14.942506722s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.072890550 (402h48m14.943973585s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:16.739208516 (402h48m16.610291551s) [upload=353639]: Next returned nil segment Dec 22 10:17:16.739225940 (402h48m16.610308985s) [upload=353639]: Waiting for error group managing segments... Dec 22 10:17:17.056843172 (402h48m16.927926217s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057237262 (402h48m16.928320307s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057619920 (402h48m16.928702965s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.071672710 (402h48m16.942755755s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.079155218 (402h48m16.950238263s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.080529355 (402h48m16.95161239s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.081308017 (402h48m16.952391062s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.087060581 (402h48m16.958143636s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.120430874 (402h48m16.991513909s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.131842193 (402h48m17.002925238s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.140731891 (402h48m17.011814936s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.325436603 (402h48m17.196519638s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339091925 (402h48m17.21017496s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339114328 (402h48m17.210197363s) [seg_pos={0 0}, upload=353639]: Segment reached optimal threshold of 13 pieces. Dec 22 10:17:17.339325330 (402h48m17.210408385s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339334477 (402h48m17.210417532s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339440859 (402h48m17.210523934s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339479453 (402h48m17.210562528s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339630210 (402h48m17.210713255s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection Dec 22 10:17:17.339642173 (402h48m17.210725208s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339663915 (402h48m17.21074695s) [seg_pos={0 0}, upload=353639]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 10:17:17.339746932 (402h48m17.210829967s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 10:17:17.339755538 (402h48m17.210838583s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 10:17:17.645053202 (402h48m17.516136237s) [upload=353639]: Done uploading segments... Dec 22 10:17:17.645073179 (402h48m17.516156234s) [upload=353639]: Upload finished. err: <nil> Duration: 5.713135966s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.767344869s trace: https://jaeger.dev.storj.io/trace/5026b732c6f0fe22 FAIL (2025-12-22 10:53:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, Piece ID: 46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ) context deadline exceeded; piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ) context deadline exceeded ======LOGS====== 22 10:17:14.811047764 (402h48m14.682130809s) [upload=353639]: Starting upload Dec 22 10:17:14.811138286 (402h48m14.682221371s) [upload=353639]: Done starting upload Dec 22 10:17:14.811261631 (402h48m14.682344696s) [upload=353639]: Uploading segments... Dec 22 10:17:14.811582702 (402h48m14.682665747s) [upload=353639]: Got next segment. Inline: false Dec 22 10:17:14.811634691 (402h48m14.682717726s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 10:17:14.811670749 (402h48m14.682753784s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 10:17:15.062219720 (402h48m14.933302775s) [seg_pos={0 0}, upload=353639]: Begin upload segment... Dec 22 10:17:15.062379986 (402h48m14.933463021s) [seg_pos={0 0}, upload=353639]: Done begin upload segment. Dec 22 10:17:15.062398501 (402h48m14.933481536s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.062447905 (402h48m14.93353095s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.063165711 (402h48m14.934248767s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.064547112 (402h48m14.935630147s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065053717 (402h48m14.936136752s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065346284 (402h48m14.936429329s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065726608 (402h48m14.936809653s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066044513 (402h48m14.937127549s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066379291 (402h48m14.937462336s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066696966 (402h48m14.937780011s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067032044 (402h48m14.938115089s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067378333 (402h48m14.938461378s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.068500340 (402h48m14.939583375s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.069610214 (402h48m14.940693259s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.071423677 (402h48m14.942506722s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.072890550 (402h48m14.943973585s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:16.739208516 (402h48m16.610291551s) [upload=353639]: Next returned nil segment Dec 22 10:17:16.739225940 (402h48m16.610308985s) [upload=353639]: Waiting for error group managing segments... Dec 22 10:17:17.056843172 (402h48m16.927926217s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057237262 (402h48m16.928320307s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057619920 (402h48m16.928702965s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.071672710 (402h48m16.942755755s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.079155218 (402h48m16.950238263s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.080529355 (402h48m16.95161239s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.081308017 (402h48m16.952391062s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.087060581 (402h48m16.958143636s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.120430874 (402h48m16.991513909s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.131842193 (402h48m17.002925238s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.140731891 (402h48m17.011814936s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.325436603 (402h48m17.196519638s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339091925 (402h48m17.21017496s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339114328 (402h48m17.210197363s) [seg_pos={0 0}, upload=353639]: Segment reached optimal threshold of 13 pieces. Dec 22 10:17:17.339325330 (402h48m17.210408385s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339334477 (402h48m17.210417532s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339440859 (402h48m17.210523934s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339479453 (402h48m17.210562528s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339630210 (402h48m17.210713255s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection Dec 22 10:17:17.339642173 (402h48m17.210725208s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339663915 (402h48m17.21074695s) [seg_pos={0 0}, upload=353639]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 10:17:17.339746932 (402h48m17.210829967s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 10:17:17.339755538 (402h48m17.210838583s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 10:17:17.645053202 (402h48m17.516136237s) [upload=353639]: Done uploading segments... Dec 22 10:17:17.645073179 (402h48m17.516156234s) [upload=353639]: Upload finished. err: <nil> Duration: 5.767421735s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 3.568278295s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 10:53:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.568290868s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.128905428s trace: https://jaeger.dev.storj.io/trace/5026b732c6f0fe22 FAIL (2025-12-22 10:52:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, Piece ID: 46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ) context deadline exceeded; piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ) context deadline exceeded ======LOGS====== 22 10:17:14.811047764 (402h48m14.682130809s) [upload=353639]: Starting upload Dec 22 10:17:14.811138286 (402h48m14.682221371s) [upload=353639]: Done starting upload Dec 22 10:17:14.811261631 (402h48m14.682344696s) [upload=353639]: Uploading segments... Dec 22 10:17:14.811582702 (402h48m14.682665747s) [upload=353639]: Got next segment. Inline: false Dec 22 10:17:14.811634691 (402h48m14.682717726s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 10:17:14.811670749 (402h48m14.682753784s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 10:17:15.062219720 (402h48m14.933302775s) [seg_pos={0 0}, upload=353639]: Begin upload segment... Dec 22 10:17:15.062379986 (402h48m14.933463021s) [seg_pos={0 0}, upload=353639]: Done begin upload segment. Dec 22 10:17:15.062398501 (402h48m14.933481536s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.062447905 (402h48m14.93353095s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.063165711 (402h48m14.934248767s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.064547112 (402h48m14.935630147s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065053717 (402h48m14.936136752s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065346284 (402h48m14.936429329s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065726608 (402h48m14.936809653s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066044513 (402h48m14.937127549s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066379291 (402h48m14.937462336s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066696966 (402h48m14.937780011s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067032044 (402h48m14.938115089s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067378333 (402h48m14.938461378s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.068500340 (402h48m14.939583375s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.069610214 (402h48m14.940693259s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.071423677 (402h48m14.942506722s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.072890550 (402h48m14.943973585s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:16.739208516 (402h48m16.610291551s) [upload=353639]: Next returned nil segment Dec 22 10:17:16.739225940 (402h48m16.610308985s) [upload=353639]: Waiting for error group managing segments... Dec 22 10:17:17.056843172 (402h48m16.927926217s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057237262 (402h48m16.928320307s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057619920 (402h48m16.928702965s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.071672710 (402h48m16.942755755s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.079155218 (402h48m16.950238263s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.080529355 (402h48m16.95161239s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.081308017 (402h48m16.952391062s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.087060581 (402h48m16.958143636s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.120430874 (402h48m16.991513909s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.131842193 (402h48m17.002925238s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.140731891 (402h48m17.011814936s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.325436603 (402h48m17.196519638s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339091925 (402h48m17.21017496s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339114328 (402h48m17.210197363s) [seg_pos={0 0}, upload=353639]: Segment reached optimal threshold of 13 pieces. Dec 22 10:17:17.339325330 (402h48m17.210408385s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339334477 (402h48m17.210417532s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339440859 (402h48m17.210523934s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339479453 (402h48m17.210562528s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339630210 (402h48m17.210713255s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection Dec 22 10:17:17.339642173 (402h48m17.210725208s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339663915 (402h48m17.21074695s) [seg_pos={0 0}, upload=353639]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 10:17:17.339746932 (402h48m17.210829967s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 10:17:17.339755538 (402h48m17.210838583s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 10:17:17.645053202 (402h48m17.516136237s) [upload=353639]: Done uploading segments... Dec 22 10:17:17.645073179 (402h48m17.516156234s) [upload=353639]: Upload finished. err: <nil> Duration: 6.128981121s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 3.697899697s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 10:52:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.69793225s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.205210527s trace: https://jaeger.dev.storj.io/trace/5026b732c6f0fe22 FAIL (2025-12-22 10:52:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, Piece ID: 46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ) context deadline exceeded; piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ) context deadline exceeded ======LOGS====== 22 10:17:14.811047764 (402h48m14.682130809s) [upload=353639]: Starting upload Dec 22 10:17:14.811138286 (402h48m14.682221371s) [upload=353639]: Done starting upload Dec 22 10:17:14.811261631 (402h48m14.682344696s) [upload=353639]: Uploading segments... Dec 22 10:17:14.811582702 (402h48m14.682665747s) [upload=353639]: Got next segment. Inline: false Dec 22 10:17:14.811634691 (402h48m14.682717726s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 10:17:14.811670749 (402h48m14.682753784s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 10:17:15.062219720 (402h48m14.933302775s) [seg_pos={0 0}, upload=353639]: Begin upload segment... Dec 22 10:17:15.062379986 (402h48m14.933463021s) [seg_pos={0 0}, upload=353639]: Done begin upload segment. Dec 22 10:17:15.062398501 (402h48m14.933481536s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.062447905 (402h48m14.93353095s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.063165711 (402h48m14.934248767s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.064547112 (402h48m14.935630147s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065053717 (402h48m14.936136752s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065346284 (402h48m14.936429329s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065726608 (402h48m14.936809653s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066044513 (402h48m14.937127549s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066379291 (402h48m14.937462336s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066696966 (402h48m14.937780011s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067032044 (402h48m14.938115089s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067378333 (402h48m14.938461378s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.068500340 (402h48m14.939583375s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.069610214 (402h48m14.940693259s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.071423677 (402h48m14.942506722s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.072890550 (402h48m14.943973585s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:16.739208516 (402h48m16.610291551s) [upload=353639]: Next returned nil segment Dec 22 10:17:16.739225940 (402h48m16.610308985s) [upload=353639]: Waiting for error group managing segments... Dec 22 10:17:17.056843172 (402h48m16.927926217s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057237262 (402h48m16.928320307s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057619920 (402h48m16.928702965s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.071672710 (402h48m16.942755755s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.079155218 (402h48m16.950238263s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.080529355 (402h48m16.95161239s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.081308017 (402h48m16.952391062s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.087060581 (402h48m16.958143636s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.120430874 (402h48m16.991513909s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.131842193 (402h48m17.002925238s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.140731891 (402h48m17.011814936s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.325436603 (402h48m17.196519638s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339091925 (402h48m17.21017496s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339114328 (402h48m17.210197363s) [seg_pos={0 0}, upload=353639]: Segment reached optimal threshold of 13 pieces. Dec 22 10:17:17.339325330 (402h48m17.210408385s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339334477 (402h48m17.210417532s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339440859 (402h48m17.210523934s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339479453 (402h48m17.210562528s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339630210 (402h48m17.210713255s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection Dec 22 10:17:17.339642173 (402h48m17.210725208s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339663915 (402h48m17.21074695s) [seg_pos={0 0}, upload=353639]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 10:17:17.339746932 (402h48m17.210829967s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 10:17:17.339755538 (402h48m17.210838583s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 10:17:17.645053202 (402h48m17.516136237s) [upload=353639]: Done uploading segments... Dec 22 10:17:17.645073179 (402h48m17.516156234s) [upload=353639]: Upload finished. err: <nil> Duration: 6.205241636s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 3.61787764s trace: https://jaeger.dev.storj.io/trace/d067e2ed6d6ba04 FAIL (2025-12-22 10:51:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.617902356s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.766350155s trace: https://jaeger.dev.storj.io/trace/5026b732c6f0fe22 FAIL (2025-12-22 10:51:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZ6d8B2pdLZntgDTzJQJzx7o7A2qYK63NQrtN5ke91nmn4idj, Piece ID: 46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ) context deadline exceeded; piecestore: (Node ID: 12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, Piece ID: 7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ) context deadline exceeded ======LOGS====== 22 10:17:14.811047764 (402h48m14.682130809s) [upload=353639]: Starting upload Dec 22 10:17:14.811138286 (402h48m14.682221371s) [upload=353639]: Done starting upload Dec 22 10:17:14.811261631 (402h48m14.682344696s) [upload=353639]: Uploading segments... Dec 22 10:17:14.811582702 (402h48m14.682665747s) [upload=353639]: Got next segment. Inline: false Dec 22 10:17:14.811634691 (402h48m14.682717726s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 22 10:17:14.811670749 (402h48m14.682753784s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 22 10:17:15.062219720 (402h48m14.933302775s) [seg_pos={0 0}, upload=353639]: Begin upload segment... Dec 22 10:17:15.062379986 (402h48m14.933463021s) [seg_pos={0 0}, upload=353639]: Done begin upload segment. Dec 22 10:17:15.062398501 (402h48m14.933481536s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.062447905 (402h48m14.93353095s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.063165711 (402h48m14.934248767s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.064547112 (402h48m14.935630147s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065053717 (402h48m14.936136752s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065346284 (402h48m14.936429329s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.065726608 (402h48m14.936809653s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066044513 (402h48m14.937127549s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066379291 (402h48m14.937462336s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.066696966 (402h48m14.937780011s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067032044 (402h48m14.938115089s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.067378333 (402h48m14.938461378s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.068500340 (402h48m14.939583375s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.069610214 (402h48m14.940693259s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.071423677 (402h48m14.942506722s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:15.072890550 (402h48m14.943973585s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Uploading piece... Dec 22 10:17:16.739208516 (402h48m16.610291551s) [upload=353639]: Next returned nil segment Dec 22 10:17:16.739225940 (402h48m16.610308985s) [upload=353639]: Waiting for error group managing segments... Dec 22 10:17:17.056843172 (402h48m16.927926217s) [noise=true , address=145.79.227.15:4875 , piece_id=2SFRFYCJEKV4F4FXJTYLX766HXUK4NM3VM2DEAF2BIAECVUIFV3Q, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057237262 (402h48m16.928320307s) [noise=true , address=145.79.227.51:4409 , piece_id=VKNU57X7XR3VC3M47HWPHS2WA6LG7IA56U5FRT4NUWDBTYSKDGDQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.057619920 (402h48m16.928702965s) [noise=true , address=145.79.227.4:4186 , piece_id=7W5NQ3PC4ND6YRIZPGT2E2BU6L2WB5MAL7LYTPYWX47ZJO2CALEA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.071672710 (402h48m16.942755755s) [noise=true , address=145.79.227.32:4835 , piece_id=SBK4NO5V7SYMDM4AGZ5ECSTUSBFZ6MXGV3U4AYO2F6O5LLT2HCQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.079155218 (402h48m16.950238263s) [noise=true , address=145.79.227.47:4265 , piece_id=SZQ5HIUDR6RICCU3ZTHBQRBZQC2LWRBK6LXPWAXC3MDRMC3WCAFA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.080529355 (402h48m16.95161239s) [noise=true , address=145.79.227.33:4870 , piece_id=NTAKE2SUCXZWXTG4QM3FKC3DVICP6O2CDDPJ24PY2JS4HPW76JWQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.081308017 (402h48m16.952391062s) [noise=true , address=145.79.227.50:4373 , piece_id=IJ4BJ42DVLYEZ4K3ATJ5MNU42CUO2MOJNYSGTM2DFQCIE4L3ZOYQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.087060581 (402h48m16.958143636s) [noise=true , address=145.79.227.43:4159 , piece_id=GVFKZ2V4KNWIR2L3QG3EGWNYGJ4THIDQZJXGRWK46BGVBGUJEXQA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.120430874 (402h48m16.991513909s) [noise=true , address=145.79.227.14:4839 , piece_id=W7TRXYZ4FSDHHBAQWOSHZDZBGXKNGRW6EGARXYQOG2ZV6HY7EMOA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.131842193 (402h48m17.002925238s) [noise=true , address=145.79.227.27:4691 , piece_id=5IUWNRS5PMMRLNAN57335LKFJLBA6F2LEDF2WB4AZWOVNQRI2BJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.140731891 (402h48m17.011814936s) [noise=true , address=145.79.227.38:4978 , piece_id=OMTED6PY4DKUHQM5XJ6YJDGEQWJAM7MCN5DFYCS664QZ222D3XKQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.325436603 (402h48m17.196519638s) [noise=true , address=45.91.116.10:4084 , piece_id=46EN2HS77EU66VIKKO35TB3ISGJTRLESS4ILLKIFE63DYGUVGVJQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339091925 (402h48m17.21017496s) [noise=true , address=77.90.31.10:4519 , piece_id=7RXCLHY4N26H5NDY7TDF37CCXQILQGJAAEBDAAKVY4BWIYPYOAOQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: <nil> Dec 22 10:17:17.339114328 (402h48m17.210197363s) [seg_pos={0 0}, upload=353639]: Segment reached optimal threshold of 13 pieces. Dec 22 10:17:17.339325330 (402h48m17.210408385s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12u52PWsW5g7XTky7sxS5mPFqpMD4AHCXFfera1WZyZF9xf6ma9): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339334477 (402h48m17.210417532s) [noise=true , address=77.90.31.14:4664 , piece_id=ZSJYUF4U77EB7TVK4XCRGTLWSSVWZOXAYFIHC5KY376ELHV5H6LQ, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339440859 (402h48m17.210523934s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EXja3EXj5TJmk2j46Bnc6TXT7N3wgxCLjzAEjbawqvtQU7BF8): protocol: expected piece hash; context canceled; EOF Dec 22 10:17:17.339479453 (402h48m17.210562528s) [noise=true , address=145.79.227.44:4193 , piece_id=RYJABC4HVBQLGAAIDT3XJK7SXAMPQUM4QNFST2YL352XCOBBE7YA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339630210 (402h48m17.210713255s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38010->45.91.116.8:4047: use of closed network connection Dec 22 10:17:17.339642173 (402h48m17.210725208s) [noise=true , address=45.91.116.8:4047 , piece_id=RQU4ZLRTNO76WTQUZJIVAMF7ISCYXHRYDEC2SOW5JZTLZ3PYI5NA, seg_pos={0 0}, upload=353639]: Long tail cancellation due to optimal threshold Dec 22 10:17:17.339663915 (402h48m17.21074695s) [seg_pos={0 0}, upload=353639]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 22 10:17:17.339746932 (402h48m17.210829967s) [upload=353639]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 22 10:17:17.339755538 (402h48m17.210838583s) [upload=353639]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 22 10:17:17.645053202 (402h48m17.516136237s) [upload=353639]: Done uploading segments... Dec 22 10:17:17.645073179 (402h48m17.516156234s) [upload=353639]: Upload finished. err: <nil> Duration: 5.766378842s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)