Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.067782307s FAIL (2025-08-07 04:59:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.067800562s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X..............................X...X..............] duration: 5.727645977s FAIL (2025-08-07 04:59:52 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.727671266s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 1.864215545s FAIL (2025-08-07 04:59:50 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.864245853s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 1.838768137s FAIL (2025-08-07 04:59:27 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.838792043s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X..............................X...X.............] duration: 5.656339145s FAIL (2025-08-07 04:59:18 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.656365596s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............................X..........X..........X......] duration: 5.712585421s FAIL (2025-08-07 04:59:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.712613275s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.13440754s FAIL (2025-08-07 04:59:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.134434031s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 1.88060255s FAIL (2025-08-07 04:58:30 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.880631304s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.199725962s FAIL (2025-08-07 04:58:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.199750618s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X..............................X...X............] duration: 5.620621311s FAIL (2025-08-07 04:58:09 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.620646237s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................................X..........X..........X.....] duration: 5.83865782s FAIL (2025-08-07 04:58:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.838675464s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................................X..........X..........X....] duration: 29.964758926s FAIL (2025-08-07 04:58:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 29.96478195s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 1.956512162s FAIL (2025-08-07 04:57:55 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.956540816s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 17.173389737s FAIL (2025-08-07 04:57:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 17.173406858s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X..............................X...X...........] duration: 5.827041826s FAIL (2025-08-07 04:57:04 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.827079168s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................................X..........X..........X...] duration: 5.784362188s FAIL (2025-08-07 04:56:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.784386977s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 1.924088036s FAIL (2025-08-07 04:56:51 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.924107965s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................X..........X..........X..] duration: 6.080696278s FAIL (2025-08-07 04:56:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 6.080730933s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X..............................X...X..........] duration: 5.595585953s FAIL (2025-08-07 04:56:35 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.59561034s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.891473928s FAIL (2025-08-07 04:56:28 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.89150654s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................X..........X..........X.] duration: 5.728757893s FAIL (2025-08-07 04:56:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.728781157s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.070441128s FAIL (2025-08-07 04:55:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.070464251s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 1.87446186s FAIL (2025-08-07 04:55:52 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.874488059s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X..............................X...X.........] duration: 5.593476551s FAIL (2025-08-07 04:55:43 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.593499364s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................................X..........X..........X] duration: 30.001891698s FAIL (2025-08-07 04:55:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 30.001917689s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.552302453s FAIL (2025-08-07 04:55:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.552502798s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X..............................X...X........] duration: 5.541659922s FAIL (2025-08-07 04:55:26 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.541679731s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X..........X..........] duration: 5.871521259s FAIL (2025-08-07 04:54:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== p3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:04.627096584 (14h39m12.565512433s) [noise=true , address=104.36.82.16:4293 , piece_id=RAUCGYLWLRML5WFEYIHTTLCQN2VACOSI6YVK7QILIJLEQZCZZZ4Q, seg_pos={0 0}, upload=14046]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:04.627136711 (14h39m12.56555256s) [seg_pos={0 0}, upload=14046]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:48:04.627218808 (14h39m12.565634658s) [upload=14046]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:48:04.627223087 (14h39m12.565638936s) [upload=14046]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:48:04.983981169 (14h39m12.922397018s) [upload=14046]: Done uploading segments... Aug 7 04:48:04.983999454 (14h39m12.922415303s) [upload=14046]: Upload finished. err: <nil> Aug 7 04:48:42.488465378 (14h39m50.426881227s) [upload=14054]: Starting upload Aug 7 04:48:42.488522797 (14h39m50.426938647s) [upload=14054]: Done starting upload Aug 7 04:48:42.488655972 (14h39m50.427071822s) [upload=14054]: Uploading segments... Aug 7 04:48:42.488722018 (14h39m50.427137857s) [upload=14054]: Got next segment. Inline: false Aug 7 04:48:42.488748830 (14h39m50.427164759s) [upload=14054]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:48:42.488753088 (14h39m50.427168937s) [upload=14054]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:48:42.768405779 (14h39m50.706821628s) [seg_pos={0 0}, upload=14054]: Begin upload segment... Aug 7 04:48:42.768450404 (14h39m50.706866253s) [seg_pos={0 0}, upload=14054]: Done begin upload segment. Aug 7 04:48:42.768463088 (14h39m50.706878927s) [noise=true , address=104.36.82.2:4730 , piece_id=LBXNDOJCZJTUO5JK4LIOXQULXBP4OYJ7U2EOHR6FAF25NVDPCCSQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768481714 (14h39m50.706897563s) [noise=true , address=45.91.116.7:4011 , piece_id=J3SAPWUNXY3VOJMT5PBRDCLTJIGQYHQLNN4RLFPLHH3P2UGTKGVA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768519626 (14h39m50.706935475s) [noise=true , address=45.91.116.6:4974 , piece_id=SAOVB5R5INZGVNDKOR5VL34FTPHKRO23PGX62ATGYEL5ZWHFVJBQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768824620 (14h39m50.707240469s) [noise=true , address=104.36.82.8:4972 , piece_id=CZEHUAO5E55TW3S6YZVYJ2QPMUXUECKAOMYXZ6KW5EXEBWQDR2VA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768971191 (14h39m50.70738704s) [noise=true , address=45.91.116.9:4263 , piece_id=6HVCX7PTMB2Q4JGCFSG33NNPYWU52PJ2O52WJDQMEBU35L7442PA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769157858 (14h39m50.707573697s) [noise=true , address=45.91.116.12:4227 , piece_id=FCXFJQKSA2VBSOSSWQCIJAVOONBFJOLRTDCLOUPFUTSOSGUVPFGQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769433395 (14h39m50.707849244s) [noise=true , address=45.91.116.9:4262 , piece_id=ERT645SMMCLX4VBSL5NBGPVKAXQJFDQJVA7267TGZA73Y63DKPIA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769676039 (14h39m50.708091888s) [noise=true , address=45.91.116.7:4010 , piece_id=Y3PQKN2FEY63GQOHAH2NUEAW3YNLM2T465LOOGE35PBBM2G6IJ7A, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769945645 (14h39m50.708361494s) [noise=true , address=45.91.116.8:4047 , piece_id=ATA5VNNB3FY6JLM7XAC6NJAXP3XEMVAB6OJGA6J2RVKJDE7KSV6Q, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770185735 (14h39m50.708601574s) [noise=true , address=104.36.82.4:4803 , piece_id=P5YNTC577545WKRAFF5ZDVOZ7BKWIBW2YEPZNYS6R64V2SIEY52A, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770394053 (14h39m50.708809902s) [noise=true , address=45.91.116.4:4939 , piece_id=BHT5K3RVVCHUNCZ3NUYKPRUG43HT7AKE2AOBHLDDYL7SEVCXD3PQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770611088 (14h39m50.709026937s) [noise=true , address=77.90.31.16:4733 , piece_id=3ELJ3SXEPZFSIYAT72DLXB6W7K6PPBVL4ZY55INTYL7KH4KM2TEQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770800280 (14h39m50.709216129s) [noise=true , address=77.90.31.14:4663 , piece_id=6LDXGGVX3WY6S27SD3THJMFS7J3QAE4HD67L45N725KPD6B47KVQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770988741 (14h39m50.70940459s) [noise=true , address=104.36.82.4:4804 , piece_id=YC2KT2WSER5L3UORMDBVIFAIMCAFRPNBIAGAUTV2JT6YNXOE7YXA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.771221355 (14h39m50.709637204s) [noise=true , address=45.91.116.5:4298 , piece_id=F7LJFPUGTAV5WYMLIZPLPDTCUUDN3PIQGNRW4Y44MWTM4ZJHW53Q, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.771423853 (14h39m50.709839692s) [noise=true , address=104.36.82.16:4295 , piece_id=6C5W5RQSSRWFRUJE6GM5ODBIHTEOCQ33V473FLF5SVWYM62ARBRQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:44.484546105 (14h39m52.422961954s) [upload=14054]: Next returned nil segment Aug 7 04:48:44.484566966 (14h39m52.422982815s) [upload=14054]: Waiting for error group managing segments... Aug 7 04:48:44.782995940 (14h39m52.721411779s) [noise=true , address=45.91.116.7:4010 , piece_id=Y3PQKN2FEY63GQOHAH2NUEAW3YNLM2T465LOOGE35PBBM2G6IJ7A, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.786326542 (14h39m52.724742391s) [noise=true , address=45.91.116.7:4011 , piece_id=J3SAPWUNXY3VOJMT5PBRDCLTJIGQYHQLNN4RLFPLHH3P2UGTKGVA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.797538894 (14h39m52.735954743s) [noise=true , address=45.91.116.9:4262 , piece_id=ERT645SMMCLX4VBSL5NBGPVKAXQJFDQJVA7267TGZA73Y63DKPIA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.798511094 (14h39m52.736926943s) [noise=true , address=77.90.31.16:4733 , piece_id=3ELJ3SXEPZFSIYAT72DLXB6W7K6PPBVL4ZY55INTYL7KH4KM2TEQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.801571149 (14h39m52.739986988s) [noise=true , address=45.91.116.4:4939 , piece_id=BHT5K3RVVCHUNCZ3NUYKPRUG43HT7AKE2AOBHLDDYL7SEVCXD3PQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.815066949 (14h39m52.753482798s) [noise=true , address=104.36.82.4:4804 , piece_id=YC2KT2WSER5L3UORMDBVIFAIMCAFRPNBIAGAUTV2JT6YNXOE7YXA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.838459695 (14h39m52.776875544s) [noise=true , address=104.36.82.4:4803 , piece_id=P5YNTC577545WKRAFF5ZDVOZ7BKWIBW2YEPZNYS6R64V2SIEY52A, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.865483037 (14h39m52.803898886s) [noise=true , address=45.91.116.8:4047 , piece_id=ATA5VNNB3FY6JLM7XAC6NJAXP3XEMVAB6OJGA6J2RVKJDE7KSV6Q, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.867563558 (14h39m52.805979407s) [noise=true , address=45.91.116.6:4974 , piece_id=SAOVB5R5INZGVNDKOR5VL34FTPHKRO23PGX62ATGYEL5ZWHFVJBQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.872889537 (14h39m52.811305376s) [noise=true , address=45.91.116.12:4227 , piece_id=FCXFJQKSA2VBSOSSWQCIJAVOONBFJOLRTDCLOUPFUTSOSGUVPFGQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.891425040 (14h39m52.829840879s) [noise=true , address=104.36.82.8:4972 , piece_id=CZEHUAO5E55TW3S6YZVYJ2QPMUXUECKAOMYXZ6KW5EXEBWQDR2VA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.892049656 (14h39m52.830465495s) [noise=true , address=77.90.31.14:4663 , piece_id=6LDXGGVX3WY6S27SD3THJMFS7J3QAE4HD67L45N725KPD6B47KVQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.895282000 (14h39m52.833697849s) [noise=true , address=45.91.116.5:4298 , piece_id=F7LJFPUGTAV5WYMLIZPLPDTCUUDN3PIQGNRW4Y44MWTM4ZJHW53Q, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.895290786 (14h39m52.833706625s) [seg_pos={0 0}, upload=14054]: Segment reached optimal threshold of 13 pieces. Aug 7 04:48:44.895375949 (14h39m52.833791798s) [noise=true , address=45.91.116.9:4263 , piece_id=6HVCX7PTMB2Q4JGCFSG33NNPYWU52PJ2O52WJDQMEBU35L7442PA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895441084 (14h39m52.833856933s) [noise=true , address=104.36.82.16:4295 , piece_id=6C5W5RQSSRWFRUJE6GM5ODBIHTEOCQ33V473FLF5SVWYM62ARBRQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895506319 (14h39m52.833922168s) [noise=true , address=104.36.82.2:4730 , piece_id=LBXNDOJCZJTUO5JK4LIOXQULXBP4OYJ7U2EOHR6FAF25NVDPCCSQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895517911 (14h39m52.83393376s) [seg_pos={0 0}, upload=14054]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:48:44.895578357 (14h39m52.833994206s) [upload=14054]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:48:44.895582434 (14h39m52.833998273s) [upload=14054]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:48:45.212908141 (14h39m53.15132399s) [upload=14054]: Done uploading segments... Aug 7 04:48:45.212928370 (14h39m53.151344219s) [upload=14054]: Upload finished. err: <nil> Duration: 5.871550174s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [X...............................X...........................] duration: 1.848396515s FAIL (2025-08-07 04:54:40 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.848420731s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.212242768s FAIL (2025-08-07 04:54:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.212262365s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X..............................X...X.......] duration: 5.58211152s FAIL (2025-08-07 04:54:17 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.582132098s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.X...............................X..........................] duration: 1.861319351s FAIL (2025-08-07 04:54:11 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.861348085s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X..........X.........] duration: 5.820565269s FAIL (2025-08-07 04:54:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== p3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:04.627096584 (14h39m12.565512433s) [noise=true , address=104.36.82.16:4293 , piece_id=RAUCGYLWLRML5WFEYIHTTLCQN2VACOSI6YVK7QILIJLEQZCZZZ4Q, seg_pos={0 0}, upload=14046]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:04.627136711 (14h39m12.56555256s) [seg_pos={0 0}, upload=14046]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:48:04.627218808 (14h39m12.565634658s) [upload=14046]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:48:04.627223087 (14h39m12.565638936s) [upload=14046]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:48:04.983981169 (14h39m12.922397018s) [upload=14046]: Done uploading segments... Aug 7 04:48:04.983999454 (14h39m12.922415303s) [upload=14046]: Upload finished. err: <nil> Aug 7 04:48:42.488465378 (14h39m50.426881227s) [upload=14054]: Starting upload Aug 7 04:48:42.488522797 (14h39m50.426938647s) [upload=14054]: Done starting upload Aug 7 04:48:42.488655972 (14h39m50.427071822s) [upload=14054]: Uploading segments... Aug 7 04:48:42.488722018 (14h39m50.427137857s) [upload=14054]: Got next segment. Inline: false Aug 7 04:48:42.488748830 (14h39m50.427164759s) [upload=14054]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:48:42.488753088 (14h39m50.427168937s) [upload=14054]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:48:42.768405779 (14h39m50.706821628s) [seg_pos={0 0}, upload=14054]: Begin upload segment... Aug 7 04:48:42.768450404 (14h39m50.706866253s) [seg_pos={0 0}, upload=14054]: Done begin upload segment. Aug 7 04:48:42.768463088 (14h39m50.706878927s) [noise=true , address=104.36.82.2:4730 , piece_id=LBXNDOJCZJTUO5JK4LIOXQULXBP4OYJ7U2EOHR6FAF25NVDPCCSQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768481714 (14h39m50.706897563s) [noise=true , address=45.91.116.7:4011 , piece_id=J3SAPWUNXY3VOJMT5PBRDCLTJIGQYHQLNN4RLFPLHH3P2UGTKGVA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768519626 (14h39m50.706935475s) [noise=true , address=45.91.116.6:4974 , piece_id=SAOVB5R5INZGVNDKOR5VL34FTPHKRO23PGX62ATGYEL5ZWHFVJBQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768824620 (14h39m50.707240469s) [noise=true , address=104.36.82.8:4972 , piece_id=CZEHUAO5E55TW3S6YZVYJ2QPMUXUECKAOMYXZ6KW5EXEBWQDR2VA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768971191 (14h39m50.70738704s) [noise=true , address=45.91.116.9:4263 , piece_id=6HVCX7PTMB2Q4JGCFSG33NNPYWU52PJ2O52WJDQMEBU35L7442PA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769157858 (14h39m50.707573697s) [noise=true , address=45.91.116.12:4227 , piece_id=FCXFJQKSA2VBSOSSWQCIJAVOONBFJOLRTDCLOUPFUTSOSGUVPFGQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769433395 (14h39m50.707849244s) [noise=true , address=45.91.116.9:4262 , piece_id=ERT645SMMCLX4VBSL5NBGPVKAXQJFDQJVA7267TGZA73Y63DKPIA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769676039 (14h39m50.708091888s) [noise=true , address=45.91.116.7:4010 , piece_id=Y3PQKN2FEY63GQOHAH2NUEAW3YNLM2T465LOOGE35PBBM2G6IJ7A, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769945645 (14h39m50.708361494s) [noise=true , address=45.91.116.8:4047 , piece_id=ATA5VNNB3FY6JLM7XAC6NJAXP3XEMVAB6OJGA6J2RVKJDE7KSV6Q, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770185735 (14h39m50.708601574s) [noise=true , address=104.36.82.4:4803 , piece_id=P5YNTC577545WKRAFF5ZDVOZ7BKWIBW2YEPZNYS6R64V2SIEY52A, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770394053 (14h39m50.708809902s) [noise=true , address=45.91.116.4:4939 , piece_id=BHT5K3RVVCHUNCZ3NUYKPRUG43HT7AKE2AOBHLDDYL7SEVCXD3PQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770611088 (14h39m50.709026937s) [noise=true , address=77.90.31.16:4733 , piece_id=3ELJ3SXEPZFSIYAT72DLXB6W7K6PPBVL4ZY55INTYL7KH4KM2TEQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770800280 (14h39m50.709216129s) [noise=true , address=77.90.31.14:4663 , piece_id=6LDXGGVX3WY6S27SD3THJMFS7J3QAE4HD67L45N725KPD6B47KVQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770988741 (14h39m50.70940459s) [noise=true , address=104.36.82.4:4804 , piece_id=YC2KT2WSER5L3UORMDBVIFAIMCAFRPNBIAGAUTV2JT6YNXOE7YXA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.771221355 (14h39m50.709637204s) [noise=true , address=45.91.116.5:4298 , piece_id=F7LJFPUGTAV5WYMLIZPLPDTCUUDN3PIQGNRW4Y44MWTM4ZJHW53Q, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.771423853 (14h39m50.709839692s) [noise=true , address=104.36.82.16:4295 , piece_id=6C5W5RQSSRWFRUJE6GM5ODBIHTEOCQ33V473FLF5SVWYM62ARBRQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:44.484546105 (14h39m52.422961954s) [upload=14054]: Next returned nil segment Aug 7 04:48:44.484566966 (14h39m52.422982815s) [upload=14054]: Waiting for error group managing segments... Aug 7 04:48:44.782995940 (14h39m52.721411779s) [noise=true , address=45.91.116.7:4010 , piece_id=Y3PQKN2FEY63GQOHAH2NUEAW3YNLM2T465LOOGE35PBBM2G6IJ7A, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.786326542 (14h39m52.724742391s) [noise=true , address=45.91.116.7:4011 , piece_id=J3SAPWUNXY3VOJMT5PBRDCLTJIGQYHQLNN4RLFPLHH3P2UGTKGVA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.797538894 (14h39m52.735954743s) [noise=true , address=45.91.116.9:4262 , piece_id=ERT645SMMCLX4VBSL5NBGPVKAXQJFDQJVA7267TGZA73Y63DKPIA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.798511094 (14h39m52.736926943s) [noise=true , address=77.90.31.16:4733 , piece_id=3ELJ3SXEPZFSIYAT72DLXB6W7K6PPBVL4ZY55INTYL7KH4KM2TEQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.801571149 (14h39m52.739986988s) [noise=true , address=45.91.116.4:4939 , piece_id=BHT5K3RVVCHUNCZ3NUYKPRUG43HT7AKE2AOBHLDDYL7SEVCXD3PQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.815066949 (14h39m52.753482798s) [noise=true , address=104.36.82.4:4804 , piece_id=YC2KT2WSER5L3UORMDBVIFAIMCAFRPNBIAGAUTV2JT6YNXOE7YXA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.838459695 (14h39m52.776875544s) [noise=true , address=104.36.82.4:4803 , piece_id=P5YNTC577545WKRAFF5ZDVOZ7BKWIBW2YEPZNYS6R64V2SIEY52A, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.865483037 (14h39m52.803898886s) [noise=true , address=45.91.116.8:4047 , piece_id=ATA5VNNB3FY6JLM7XAC6NJAXP3XEMVAB6OJGA6J2RVKJDE7KSV6Q, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.867563558 (14h39m52.805979407s) [noise=true , address=45.91.116.6:4974 , piece_id=SAOVB5R5INZGVNDKOR5VL34FTPHKRO23PGX62ATGYEL5ZWHFVJBQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.872889537 (14h39m52.811305376s) [noise=true , address=45.91.116.12:4227 , piece_id=FCXFJQKSA2VBSOSSWQCIJAVOONBFJOLRTDCLOUPFUTSOSGUVPFGQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.891425040 (14h39m52.829840879s) [noise=true , address=104.36.82.8:4972 , piece_id=CZEHUAO5E55TW3S6YZVYJ2QPMUXUECKAOMYXZ6KW5EXEBWQDR2VA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.892049656 (14h39m52.830465495s) [noise=true , address=77.90.31.14:4663 , piece_id=6LDXGGVX3WY6S27SD3THJMFS7J3QAE4HD67L45N725KPD6B47KVQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.895282000 (14h39m52.833697849s) [noise=true , address=45.91.116.5:4298 , piece_id=F7LJFPUGTAV5WYMLIZPLPDTCUUDN3PIQGNRW4Y44MWTM4ZJHW53Q, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.895290786 (14h39m52.833706625s) [seg_pos={0 0}, upload=14054]: Segment reached optimal threshold of 13 pieces. Aug 7 04:48:44.895375949 (14h39m52.833791798s) [noise=true , address=45.91.116.9:4263 , piece_id=6HVCX7PTMB2Q4JGCFSG33NNPYWU52PJ2O52WJDQMEBU35L7442PA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895441084 (14h39m52.833856933s) [noise=true , address=104.36.82.16:4295 , piece_id=6C5W5RQSSRWFRUJE6GM5ODBIHTEOCQ33V473FLF5SVWYM62ARBRQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895506319 (14h39m52.833922168s) [noise=true , address=104.36.82.2:4730 , piece_id=LBXNDOJCZJTUO5JK4LIOXQULXBP4OYJ7U2EOHR6FAF25NVDPCCSQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895517911 (14h39m52.83393376s) [seg_pos={0 0}, upload=14054]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:48:44.895578357 (14h39m52.833994206s) [upload=14054]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:48:44.895582434 (14h39m52.833998273s) [upload=14054]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:48:45.212908141 (14h39m53.15132399s) [upload=14054]: Done uploading segments... Aug 7 04:48:45.212928370 (14h39m53.151344219s) [upload=14054]: Upload finished. err: <nil> Duration: 5.820590808s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 16.259319626s FAIL (2025-08-07 04:53:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 16.259338733s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X..........X........] duration: 5.735961823s FAIL (2025-08-07 04:53:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== p3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:04.627096584 (14h39m12.565512433s) [noise=true , address=104.36.82.16:4293 , piece_id=RAUCGYLWLRML5WFEYIHTTLCQN2VACOSI6YVK7QILIJLEQZCZZZ4Q, seg_pos={0 0}, upload=14046]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:04.627136711 (14h39m12.56555256s) [seg_pos={0 0}, upload=14046]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:48:04.627218808 (14h39m12.565634658s) [upload=14046]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:48:04.627223087 (14h39m12.565638936s) [upload=14046]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:48:04.983981169 (14h39m12.922397018s) [upload=14046]: Done uploading segments... Aug 7 04:48:04.983999454 (14h39m12.922415303s) [upload=14046]: Upload finished. err: <nil> Aug 7 04:48:42.488465378 (14h39m50.426881227s) [upload=14054]: Starting upload Aug 7 04:48:42.488522797 (14h39m50.426938647s) [upload=14054]: Done starting upload Aug 7 04:48:42.488655972 (14h39m50.427071822s) [upload=14054]: Uploading segments... Aug 7 04:48:42.488722018 (14h39m50.427137857s) [upload=14054]: Got next segment. Inline: false Aug 7 04:48:42.488748830 (14h39m50.427164759s) [upload=14054]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:48:42.488753088 (14h39m50.427168937s) [upload=14054]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:48:42.768405779 (14h39m50.706821628s) [seg_pos={0 0}, upload=14054]: Begin upload segment... Aug 7 04:48:42.768450404 (14h39m50.706866253s) [seg_pos={0 0}, upload=14054]: Done begin upload segment. Aug 7 04:48:42.768463088 (14h39m50.706878927s) [noise=true , address=104.36.82.2:4730 , piece_id=LBXNDOJCZJTUO5JK4LIOXQULXBP4OYJ7U2EOHR6FAF25NVDPCCSQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768481714 (14h39m50.706897563s) [noise=true , address=45.91.116.7:4011 , piece_id=J3SAPWUNXY3VOJMT5PBRDCLTJIGQYHQLNN4RLFPLHH3P2UGTKGVA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768519626 (14h39m50.706935475s) [noise=true , address=45.91.116.6:4974 , piece_id=SAOVB5R5INZGVNDKOR5VL34FTPHKRO23PGX62ATGYEL5ZWHFVJBQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768824620 (14h39m50.707240469s) [noise=true , address=104.36.82.8:4972 , piece_id=CZEHUAO5E55TW3S6YZVYJ2QPMUXUECKAOMYXZ6KW5EXEBWQDR2VA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768971191 (14h39m50.70738704s) [noise=true , address=45.91.116.9:4263 , piece_id=6HVCX7PTMB2Q4JGCFSG33NNPYWU52PJ2O52WJDQMEBU35L7442PA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769157858 (14h39m50.707573697s) [noise=true , address=45.91.116.12:4227 , piece_id=FCXFJQKSA2VBSOSSWQCIJAVOONBFJOLRTDCLOUPFUTSOSGUVPFGQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769433395 (14h39m50.707849244s) [noise=true , address=45.91.116.9:4262 , piece_id=ERT645SMMCLX4VBSL5NBGPVKAXQJFDQJVA7267TGZA73Y63DKPIA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769676039 (14h39m50.708091888s) [noise=true , address=45.91.116.7:4010 , piece_id=Y3PQKN2FEY63GQOHAH2NUEAW3YNLM2T465LOOGE35PBBM2G6IJ7A, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769945645 (14h39m50.708361494s) [noise=true , address=45.91.116.8:4047 , piece_id=ATA5VNNB3FY6JLM7XAC6NJAXP3XEMVAB6OJGA6J2RVKJDE7KSV6Q, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770185735 (14h39m50.708601574s) [noise=true , address=104.36.82.4:4803 , piece_id=P5YNTC577545WKRAFF5ZDVOZ7BKWIBW2YEPZNYS6R64V2SIEY52A, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770394053 (14h39m50.708809902s) [noise=true , address=45.91.116.4:4939 , piece_id=BHT5K3RVVCHUNCZ3NUYKPRUG43HT7AKE2AOBHLDDYL7SEVCXD3PQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770611088 (14h39m50.709026937s) [noise=true , address=77.90.31.16:4733 , piece_id=3ELJ3SXEPZFSIYAT72DLXB6W7K6PPBVL4ZY55INTYL7KH4KM2TEQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770800280 (14h39m50.709216129s) [noise=true , address=77.90.31.14:4663 , piece_id=6LDXGGVX3WY6S27SD3THJMFS7J3QAE4HD67L45N725KPD6B47KVQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770988741 (14h39m50.70940459s) [noise=true , address=104.36.82.4:4804 , piece_id=YC2KT2WSER5L3UORMDBVIFAIMCAFRPNBIAGAUTV2JT6YNXOE7YXA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.771221355 (14h39m50.709637204s) [noise=true , address=45.91.116.5:4298 , piece_id=F7LJFPUGTAV5WYMLIZPLPDTCUUDN3PIQGNRW4Y44MWTM4ZJHW53Q, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.771423853 (14h39m50.709839692s) [noise=true , address=104.36.82.16:4295 , piece_id=6C5W5RQSSRWFRUJE6GM5ODBIHTEOCQ33V473FLF5SVWYM62ARBRQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:44.484546105 (14h39m52.422961954s) [upload=14054]: Next returned nil segment Aug 7 04:48:44.484566966 (14h39m52.422982815s) [upload=14054]: Waiting for error group managing segments... Aug 7 04:48:44.782995940 (14h39m52.721411779s) [noise=true , address=45.91.116.7:4010 , piece_id=Y3PQKN2FEY63GQOHAH2NUEAW3YNLM2T465LOOGE35PBBM2G6IJ7A, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.786326542 (14h39m52.724742391s) [noise=true , address=45.91.116.7:4011 , piece_id=J3SAPWUNXY3VOJMT5PBRDCLTJIGQYHQLNN4RLFPLHH3P2UGTKGVA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.797538894 (14h39m52.735954743s) [noise=true , address=45.91.116.9:4262 , piece_id=ERT645SMMCLX4VBSL5NBGPVKAXQJFDQJVA7267TGZA73Y63DKPIA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.798511094 (14h39m52.736926943s) [noise=true , address=77.90.31.16:4733 , piece_id=3ELJ3SXEPZFSIYAT72DLXB6W7K6PPBVL4ZY55INTYL7KH4KM2TEQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.801571149 (14h39m52.739986988s) [noise=true , address=45.91.116.4:4939 , piece_id=BHT5K3RVVCHUNCZ3NUYKPRUG43HT7AKE2AOBHLDDYL7SEVCXD3PQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.815066949 (14h39m52.753482798s) [noise=true , address=104.36.82.4:4804 , piece_id=YC2KT2WSER5L3UORMDBVIFAIMCAFRPNBIAGAUTV2JT6YNXOE7YXA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.838459695 (14h39m52.776875544s) [noise=true , address=104.36.82.4:4803 , piece_id=P5YNTC577545WKRAFF5ZDVOZ7BKWIBW2YEPZNYS6R64V2SIEY52A, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.865483037 (14h39m52.803898886s) [noise=true , address=45.91.116.8:4047 , piece_id=ATA5VNNB3FY6JLM7XAC6NJAXP3XEMVAB6OJGA6J2RVKJDE7KSV6Q, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.867563558 (14h39m52.805979407s) [noise=true , address=45.91.116.6:4974 , piece_id=SAOVB5R5INZGVNDKOR5VL34FTPHKRO23PGX62ATGYEL5ZWHFVJBQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.872889537 (14h39m52.811305376s) [noise=true , address=45.91.116.12:4227 , piece_id=FCXFJQKSA2VBSOSSWQCIJAVOONBFJOLRTDCLOUPFUTSOSGUVPFGQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.891425040 (14h39m52.829840879s) [noise=true , address=104.36.82.8:4972 , piece_id=CZEHUAO5E55TW3S6YZVYJ2QPMUXUECKAOMYXZ6KW5EXEBWQDR2VA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.892049656 (14h39m52.830465495s) [noise=true , address=77.90.31.14:4663 , piece_id=6LDXGGVX3WY6S27SD3THJMFS7J3QAE4HD67L45N725KPD6B47KVQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.895282000 (14h39m52.833697849s) [noise=true , address=45.91.116.5:4298 , piece_id=F7LJFPUGTAV5WYMLIZPLPDTCUUDN3PIQGNRW4Y44MWTM4ZJHW53Q, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.895290786 (14h39m52.833706625s) [seg_pos={0 0}, upload=14054]: Segment reached optimal threshold of 13 pieces. Aug 7 04:48:44.895375949 (14h39m52.833791798s) [noise=true , address=45.91.116.9:4263 , piece_id=6HVCX7PTMB2Q4JGCFSG33NNPYWU52PJ2O52WJDQMEBU35L7442PA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895441084 (14h39m52.833856933s) [noise=true , address=104.36.82.16:4295 , piece_id=6C5W5RQSSRWFRUJE6GM5ODBIHTEOCQ33V473FLF5SVWYM62ARBRQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895506319 (14h39m52.833922168s) [noise=true , address=104.36.82.2:4730 , piece_id=LBXNDOJCZJTUO5JK4LIOXQULXBP4OYJ7U2EOHR6FAF25NVDPCCSQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895517911 (14h39m52.83393376s) [seg_pos={0 0}, upload=14054]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:48:44.895578357 (14h39m52.833994206s) [upload=14054]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:48:44.895582434 (14h39m52.833998273s) [upload=14054]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:48:45.212908141 (14h39m53.15132399s) [upload=14054]: Done uploading segments... Aug 7 04:48:45.212928370 (14h39m53.151344219s) [upload=14054]: Upload finished. err: <nil> Duration: 5.73598651s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..X...............................X.........................] duration: 2.087540841s FAIL (2025-08-07 04:53:32 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 2.087570328s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X..............................X...X......] duration: 5.589723668s FAIL (2025-08-07 04:53:18 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.589852234s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X..........X.......] duration: 6.386436014s FAIL (2025-08-07 04:53:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== p3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:04.627096584 (14h39m12.565512433s) [noise=true , address=104.36.82.16:4293 , piece_id=RAUCGYLWLRML5WFEYIHTTLCQN2VACOSI6YVK7QILIJLEQZCZZZ4Q, seg_pos={0 0}, upload=14046]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:04.627136711 (14h39m12.56555256s) [seg_pos={0 0}, upload=14046]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:48:04.627218808 (14h39m12.565634658s) [upload=14046]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:48:04.627223087 (14h39m12.565638936s) [upload=14046]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:48:04.983981169 (14h39m12.922397018s) [upload=14046]: Done uploading segments... Aug 7 04:48:04.983999454 (14h39m12.922415303s) [upload=14046]: Upload finished. err: <nil> Aug 7 04:48:42.488465378 (14h39m50.426881227s) [upload=14054]: Starting upload Aug 7 04:48:42.488522797 (14h39m50.426938647s) [upload=14054]: Done starting upload Aug 7 04:48:42.488655972 (14h39m50.427071822s) [upload=14054]: Uploading segments... Aug 7 04:48:42.488722018 (14h39m50.427137857s) [upload=14054]: Got next segment. Inline: false Aug 7 04:48:42.488748830 (14h39m50.427164759s) [upload=14054]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:48:42.488753088 (14h39m50.427168937s) [upload=14054]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:48:42.768405779 (14h39m50.706821628s) [seg_pos={0 0}, upload=14054]: Begin upload segment... Aug 7 04:48:42.768450404 (14h39m50.706866253s) [seg_pos={0 0}, upload=14054]: Done begin upload segment. Aug 7 04:48:42.768463088 (14h39m50.706878927s) [noise=true , address=104.36.82.2:4730 , piece_id=LBXNDOJCZJTUO5JK4LIOXQULXBP4OYJ7U2EOHR6FAF25NVDPCCSQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768481714 (14h39m50.706897563s) [noise=true , address=45.91.116.7:4011 , piece_id=J3SAPWUNXY3VOJMT5PBRDCLTJIGQYHQLNN4RLFPLHH3P2UGTKGVA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768519626 (14h39m50.706935475s) [noise=true , address=45.91.116.6:4974 , piece_id=SAOVB5R5INZGVNDKOR5VL34FTPHKRO23PGX62ATGYEL5ZWHFVJBQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768824620 (14h39m50.707240469s) [noise=true , address=104.36.82.8:4972 , piece_id=CZEHUAO5E55TW3S6YZVYJ2QPMUXUECKAOMYXZ6KW5EXEBWQDR2VA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768971191 (14h39m50.70738704s) [noise=true , address=45.91.116.9:4263 , piece_id=6HVCX7PTMB2Q4JGCFSG33NNPYWU52PJ2O52WJDQMEBU35L7442PA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769157858 (14h39m50.707573697s) [noise=true , address=45.91.116.12:4227 , piece_id=FCXFJQKSA2VBSOSSWQCIJAVOONBFJOLRTDCLOUPFUTSOSGUVPFGQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769433395 (14h39m50.707849244s) [noise=true , address=45.91.116.9:4262 , piece_id=ERT645SMMCLX4VBSL5NBGPVKAXQJFDQJVA7267TGZA73Y63DKPIA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769676039 (14h39m50.708091888s) [noise=true , address=45.91.116.7:4010 , piece_id=Y3PQKN2FEY63GQOHAH2NUEAW3YNLM2T465LOOGE35PBBM2G6IJ7A, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769945645 (14h39m50.708361494s) [noise=true , address=45.91.116.8:4047 , piece_id=ATA5VNNB3FY6JLM7XAC6NJAXP3XEMVAB6OJGA6J2RVKJDE7KSV6Q, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770185735 (14h39m50.708601574s) [noise=true , address=104.36.82.4:4803 , piece_id=P5YNTC577545WKRAFF5ZDVOZ7BKWIBW2YEPZNYS6R64V2SIEY52A, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770394053 (14h39m50.708809902s) [noise=true , address=45.91.116.4:4939 , piece_id=BHT5K3RVVCHUNCZ3NUYKPRUG43HT7AKE2AOBHLDDYL7SEVCXD3PQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770611088 (14h39m50.709026937s) [noise=true , address=77.90.31.16:4733 , piece_id=3ELJ3SXEPZFSIYAT72DLXB6W7K6PPBVL4ZY55INTYL7KH4KM2TEQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770800280 (14h39m50.709216129s) [noise=true , address=77.90.31.14:4663 , piece_id=6LDXGGVX3WY6S27SD3THJMFS7J3QAE4HD67L45N725KPD6B47KVQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770988741 (14h39m50.70940459s) [noise=true , address=104.36.82.4:4804 , piece_id=YC2KT2WSER5L3UORMDBVIFAIMCAFRPNBIAGAUTV2JT6YNXOE7YXA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.771221355 (14h39m50.709637204s) [noise=true , address=45.91.116.5:4298 , piece_id=F7LJFPUGTAV5WYMLIZPLPDTCUUDN3PIQGNRW4Y44MWTM4ZJHW53Q, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.771423853 (14h39m50.709839692s) [noise=true , address=104.36.82.16:4295 , piece_id=6C5W5RQSSRWFRUJE6GM5ODBIHTEOCQ33V473FLF5SVWYM62ARBRQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:44.484546105 (14h39m52.422961954s) [upload=14054]: Next returned nil segment Aug 7 04:48:44.484566966 (14h39m52.422982815s) [upload=14054]: Waiting for error group managing segments... Aug 7 04:48:44.782995940 (14h39m52.721411779s) [noise=true , address=45.91.116.7:4010 , piece_id=Y3PQKN2FEY63GQOHAH2NUEAW3YNLM2T465LOOGE35PBBM2G6IJ7A, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.786326542 (14h39m52.724742391s) [noise=true , address=45.91.116.7:4011 , piece_id=J3SAPWUNXY3VOJMT5PBRDCLTJIGQYHQLNN4RLFPLHH3P2UGTKGVA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.797538894 (14h39m52.735954743s) [noise=true , address=45.91.116.9:4262 , piece_id=ERT645SMMCLX4VBSL5NBGPVKAXQJFDQJVA7267TGZA73Y63DKPIA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.798511094 (14h39m52.736926943s) [noise=true , address=77.90.31.16:4733 , piece_id=3ELJ3SXEPZFSIYAT72DLXB6W7K6PPBVL4ZY55INTYL7KH4KM2TEQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.801571149 (14h39m52.739986988s) [noise=true , address=45.91.116.4:4939 , piece_id=BHT5K3RVVCHUNCZ3NUYKPRUG43HT7AKE2AOBHLDDYL7SEVCXD3PQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.815066949 (14h39m52.753482798s) [noise=true , address=104.36.82.4:4804 , piece_id=YC2KT2WSER5L3UORMDBVIFAIMCAFRPNBIAGAUTV2JT6YNXOE7YXA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.838459695 (14h39m52.776875544s) [noise=true , address=104.36.82.4:4803 , piece_id=P5YNTC577545WKRAFF5ZDVOZ7BKWIBW2YEPZNYS6R64V2SIEY52A, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.865483037 (14h39m52.803898886s) [noise=true , address=45.91.116.8:4047 , piece_id=ATA5VNNB3FY6JLM7XAC6NJAXP3XEMVAB6OJGA6J2RVKJDE7KSV6Q, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.867563558 (14h39m52.805979407s) [noise=true , address=45.91.116.6:4974 , piece_id=SAOVB5R5INZGVNDKOR5VL34FTPHKRO23PGX62ATGYEL5ZWHFVJBQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.872889537 (14h39m52.811305376s) [noise=true , address=45.91.116.12:4227 , piece_id=FCXFJQKSA2VBSOSSWQCIJAVOONBFJOLRTDCLOUPFUTSOSGUVPFGQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.891425040 (14h39m52.829840879s) [noise=true , address=104.36.82.8:4972 , piece_id=CZEHUAO5E55TW3S6YZVYJ2QPMUXUECKAOMYXZ6KW5EXEBWQDR2VA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.892049656 (14h39m52.830465495s) [noise=true , address=77.90.31.14:4663 , piece_id=6LDXGGVX3WY6S27SD3THJMFS7J3QAE4HD67L45N725KPD6B47KVQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.895282000 (14h39m52.833697849s) [noise=true , address=45.91.116.5:4298 , piece_id=F7LJFPUGTAV5WYMLIZPLPDTCUUDN3PIQGNRW4Y44MWTM4ZJHW53Q, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.895290786 (14h39m52.833706625s) [seg_pos={0 0}, upload=14054]: Segment reached optimal threshold of 13 pieces. Aug 7 04:48:44.895375949 (14h39m52.833791798s) [noise=true , address=45.91.116.9:4263 , piece_id=6HVCX7PTMB2Q4JGCFSG33NNPYWU52PJ2O52WJDQMEBU35L7442PA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895441084 (14h39m52.833856933s) [noise=true , address=104.36.82.16:4295 , piece_id=6C5W5RQSSRWFRUJE6GM5ODBIHTEOCQ33V473FLF5SVWYM62ARBRQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895506319 (14h39m52.833922168s) [noise=true , address=104.36.82.2:4730 , piece_id=LBXNDOJCZJTUO5JK4LIOXQULXBP4OYJ7U2EOHR6FAF25NVDPCCSQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895517911 (14h39m52.83393376s) [seg_pos={0 0}, upload=14054]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:48:44.895578357 (14h39m52.833994206s) [upload=14054]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:48:44.895582434 (14h39m52.833998273s) [upload=14054]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:48:45.212908141 (14h39m53.15132399s) [upload=14054]: Done uploading segments... Aug 7 04:48:45.212928370 (14h39m53.151344219s) [upload=14054]: Upload finished. err: <nil> Duration: 6.386462896s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...X...............................X........................] duration: 1.864118546s FAIL (2025-08-07 04:53:00 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.864153622s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X..............................X...X.....] duration: 6.367750511s FAIL (2025-08-07 04:52:55 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 6.367778243s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X..........X......] duration: 13.860253113s FAIL (2025-08-07 04:52:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== p3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:04.627096584 (14h39m12.565512433s) [noise=true , address=104.36.82.16:4293 , piece_id=RAUCGYLWLRML5WFEYIHTTLCQN2VACOSI6YVK7QILIJLEQZCZZZ4Q, seg_pos={0 0}, upload=14046]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:04.627136711 (14h39m12.56555256s) [seg_pos={0 0}, upload=14046]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:48:04.627218808 (14h39m12.565634658s) [upload=14046]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:48:04.627223087 (14h39m12.565638936s) [upload=14046]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:48:04.983981169 (14h39m12.922397018s) [upload=14046]: Done uploading segments... Aug 7 04:48:04.983999454 (14h39m12.922415303s) [upload=14046]: Upload finished. err: <nil> Aug 7 04:48:42.488465378 (14h39m50.426881227s) [upload=14054]: Starting upload Aug 7 04:48:42.488522797 (14h39m50.426938647s) [upload=14054]: Done starting upload Aug 7 04:48:42.488655972 (14h39m50.427071822s) [upload=14054]: Uploading segments... Aug 7 04:48:42.488722018 (14h39m50.427137857s) [upload=14054]: Got next segment. Inline: false Aug 7 04:48:42.488748830 (14h39m50.427164759s) [upload=14054]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:48:42.488753088 (14h39m50.427168937s) [upload=14054]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:48:42.768405779 (14h39m50.706821628s) [seg_pos={0 0}, upload=14054]: Begin upload segment... Aug 7 04:48:42.768450404 (14h39m50.706866253s) [seg_pos={0 0}, upload=14054]: Done begin upload segment. Aug 7 04:48:42.768463088 (14h39m50.706878927s) [noise=true , address=104.36.82.2:4730 , piece_id=LBXNDOJCZJTUO5JK4LIOXQULXBP4OYJ7U2EOHR6FAF25NVDPCCSQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768481714 (14h39m50.706897563s) [noise=true , address=45.91.116.7:4011 , piece_id=J3SAPWUNXY3VOJMT5PBRDCLTJIGQYHQLNN4RLFPLHH3P2UGTKGVA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768519626 (14h39m50.706935475s) [noise=true , address=45.91.116.6:4974 , piece_id=SAOVB5R5INZGVNDKOR5VL34FTPHKRO23PGX62ATGYEL5ZWHFVJBQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768824620 (14h39m50.707240469s) [noise=true , address=104.36.82.8:4972 , piece_id=CZEHUAO5E55TW3S6YZVYJ2QPMUXUECKAOMYXZ6KW5EXEBWQDR2VA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768971191 (14h39m50.70738704s) [noise=true , address=45.91.116.9:4263 , piece_id=6HVCX7PTMB2Q4JGCFSG33NNPYWU52PJ2O52WJDQMEBU35L7442PA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769157858 (14h39m50.707573697s) [noise=true , address=45.91.116.12:4227 , piece_id=FCXFJQKSA2VBSOSSWQCIJAVOONBFJOLRTDCLOUPFUTSOSGUVPFGQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769433395 (14h39m50.707849244s) [noise=true , address=45.91.116.9:4262 , piece_id=ERT645SMMCLX4VBSL5NBGPVKAXQJFDQJVA7267TGZA73Y63DKPIA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769676039 (14h39m50.708091888s) [noise=true , address=45.91.116.7:4010 , piece_id=Y3PQKN2FEY63GQOHAH2NUEAW3YNLM2T465LOOGE35PBBM2G6IJ7A, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769945645 (14h39m50.708361494s) [noise=true , address=45.91.116.8:4047 , piece_id=ATA5VNNB3FY6JLM7XAC6NJAXP3XEMVAB6OJGA6J2RVKJDE7KSV6Q, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770185735 (14h39m50.708601574s) [noise=true , address=104.36.82.4:4803 , piece_id=P5YNTC577545WKRAFF5ZDVOZ7BKWIBW2YEPZNYS6R64V2SIEY52A, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770394053 (14h39m50.708809902s) [noise=true , address=45.91.116.4:4939 , piece_id=BHT5K3RVVCHUNCZ3NUYKPRUG43HT7AKE2AOBHLDDYL7SEVCXD3PQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770611088 (14h39m50.709026937s) [noise=true , address=77.90.31.16:4733 , piece_id=3ELJ3SXEPZFSIYAT72DLXB6W7K6PPBVL4ZY55INTYL7KH4KM2TEQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770800280 (14h39m50.709216129s) [noise=true , address=77.90.31.14:4663 , piece_id=6LDXGGVX3WY6S27SD3THJMFS7J3QAE4HD67L45N725KPD6B47KVQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770988741 (14h39m50.70940459s) [noise=true , address=104.36.82.4:4804 , piece_id=YC2KT2WSER5L3UORMDBVIFAIMCAFRPNBIAGAUTV2JT6YNXOE7YXA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.771221355 (14h39m50.709637204s) [noise=true , address=45.91.116.5:4298 , piece_id=F7LJFPUGTAV5WYMLIZPLPDTCUUDN3PIQGNRW4Y44MWTM4ZJHW53Q, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.771423853 (14h39m50.709839692s) [noise=true , address=104.36.82.16:4295 , piece_id=6C5W5RQSSRWFRUJE6GM5ODBIHTEOCQ33V473FLF5SVWYM62ARBRQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:44.484546105 (14h39m52.422961954s) [upload=14054]: Next returned nil segment Aug 7 04:48:44.484566966 (14h39m52.422982815s) [upload=14054]: Waiting for error group managing segments... Aug 7 04:48:44.782995940 (14h39m52.721411779s) [noise=true , address=45.91.116.7:4010 , piece_id=Y3PQKN2FEY63GQOHAH2NUEAW3YNLM2T465LOOGE35PBBM2G6IJ7A, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.786326542 (14h39m52.724742391s) [noise=true , address=45.91.116.7:4011 , piece_id=J3SAPWUNXY3VOJMT5PBRDCLTJIGQYHQLNN4RLFPLHH3P2UGTKGVA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.797538894 (14h39m52.735954743s) [noise=true , address=45.91.116.9:4262 , piece_id=ERT645SMMCLX4VBSL5NBGPVKAXQJFDQJVA7267TGZA73Y63DKPIA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.798511094 (14h39m52.736926943s) [noise=true , address=77.90.31.16:4733 , piece_id=3ELJ3SXEPZFSIYAT72DLXB6W7K6PPBVL4ZY55INTYL7KH4KM2TEQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.801571149 (14h39m52.739986988s) [noise=true , address=45.91.116.4:4939 , piece_id=BHT5K3RVVCHUNCZ3NUYKPRUG43HT7AKE2AOBHLDDYL7SEVCXD3PQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.815066949 (14h39m52.753482798s) [noise=true , address=104.36.82.4:4804 , piece_id=YC2KT2WSER5L3UORMDBVIFAIMCAFRPNBIAGAUTV2JT6YNXOE7YXA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.838459695 (14h39m52.776875544s) [noise=true , address=104.36.82.4:4803 , piece_id=P5YNTC577545WKRAFF5ZDVOZ7BKWIBW2YEPZNYS6R64V2SIEY52A, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.865483037 (14h39m52.803898886s) [noise=true , address=45.91.116.8:4047 , piece_id=ATA5VNNB3FY6JLM7XAC6NJAXP3XEMVAB6OJGA6J2RVKJDE7KSV6Q, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.867563558 (14h39m52.805979407s) [noise=true , address=45.91.116.6:4974 , piece_id=SAOVB5R5INZGVNDKOR5VL34FTPHKRO23PGX62ATGYEL5ZWHFVJBQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.872889537 (14h39m52.811305376s) [noise=true , address=45.91.116.12:4227 , piece_id=FCXFJQKSA2VBSOSSWQCIJAVOONBFJOLRTDCLOUPFUTSOSGUVPFGQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.891425040 (14h39m52.829840879s) [noise=true , address=104.36.82.8:4972 , piece_id=CZEHUAO5E55TW3S6YZVYJ2QPMUXUECKAOMYXZ6KW5EXEBWQDR2VA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.892049656 (14h39m52.830465495s) [noise=true , address=77.90.31.14:4663 , piece_id=6LDXGGVX3WY6S27SD3THJMFS7J3QAE4HD67L45N725KPD6B47KVQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.895282000 (14h39m52.833697849s) [noise=true , address=45.91.116.5:4298 , piece_id=F7LJFPUGTAV5WYMLIZPLPDTCUUDN3PIQGNRW4Y44MWTM4ZJHW53Q, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.895290786 (14h39m52.833706625s) [seg_pos={0 0}, upload=14054]: Segment reached optimal threshold of 13 pieces. Aug 7 04:48:44.895375949 (14h39m52.833791798s) [noise=true , address=45.91.116.9:4263 , piece_id=6HVCX7PTMB2Q4JGCFSG33NNPYWU52PJ2O52WJDQMEBU35L7442PA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895441084 (14h39m52.833856933s) [noise=true , address=104.36.82.16:4295 , piece_id=6C5W5RQSSRWFRUJE6GM5ODBIHTEOCQ33V473FLF5SVWYM62ARBRQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895506319 (14h39m52.833922168s) [noise=true , address=104.36.82.2:4730 , piece_id=LBXNDOJCZJTUO5JK4LIOXQULXBP4OYJ7U2EOHR6FAF25NVDPCCSQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895517911 (14h39m52.83393376s) [seg_pos={0 0}, upload=14054]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:48:44.895578357 (14h39m52.833994206s) [upload=14054]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:48:44.895582434 (14h39m52.833998273s) [upload=14054]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:48:45.212908141 (14h39m53.15132399s) [upload=14054]: Done uploading segments... Aug 7 04:48:45.212928370 (14h39m53.151344219s) [upload=14054]: Upload finished. err: <nil> Duration: 13.860279303s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.133877783s FAIL (2025-08-07 04:52:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.133901287s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X..........X.....] duration: 5.870998009s FAIL (2025-08-07 04:52:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== p3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:04.627096584 (14h39m12.565512433s) [noise=true , address=104.36.82.16:4293 , piece_id=RAUCGYLWLRML5WFEYIHTTLCQN2VACOSI6YVK7QILIJLEQZCZZZ4Q, seg_pos={0 0}, upload=14046]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:04.627136711 (14h39m12.56555256s) [seg_pos={0 0}, upload=14046]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:48:04.627218808 (14h39m12.565634658s) [upload=14046]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:48:04.627223087 (14h39m12.565638936s) [upload=14046]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:48:04.983981169 (14h39m12.922397018s) [upload=14046]: Done uploading segments... Aug 7 04:48:04.983999454 (14h39m12.922415303s) [upload=14046]: Upload finished. err: <nil> Aug 7 04:48:42.488465378 (14h39m50.426881227s) [upload=14054]: Starting upload Aug 7 04:48:42.488522797 (14h39m50.426938647s) [upload=14054]: Done starting upload Aug 7 04:48:42.488655972 (14h39m50.427071822s) [upload=14054]: Uploading segments... Aug 7 04:48:42.488722018 (14h39m50.427137857s) [upload=14054]: Got next segment. Inline: false Aug 7 04:48:42.488748830 (14h39m50.427164759s) [upload=14054]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:48:42.488753088 (14h39m50.427168937s) [upload=14054]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:48:42.768405779 (14h39m50.706821628s) [seg_pos={0 0}, upload=14054]: Begin upload segment... Aug 7 04:48:42.768450404 (14h39m50.706866253s) [seg_pos={0 0}, upload=14054]: Done begin upload segment. Aug 7 04:48:42.768463088 (14h39m50.706878927s) [noise=true , address=104.36.82.2:4730 , piece_id=LBXNDOJCZJTUO5JK4LIOXQULXBP4OYJ7U2EOHR6FAF25NVDPCCSQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768481714 (14h39m50.706897563s) [noise=true , address=45.91.116.7:4011 , piece_id=J3SAPWUNXY3VOJMT5PBRDCLTJIGQYHQLNN4RLFPLHH3P2UGTKGVA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768519626 (14h39m50.706935475s) [noise=true , address=45.91.116.6:4974 , piece_id=SAOVB5R5INZGVNDKOR5VL34FTPHKRO23PGX62ATGYEL5ZWHFVJBQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768824620 (14h39m50.707240469s) [noise=true , address=104.36.82.8:4972 , piece_id=CZEHUAO5E55TW3S6YZVYJ2QPMUXUECKAOMYXZ6KW5EXEBWQDR2VA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768971191 (14h39m50.70738704s) [noise=true , address=45.91.116.9:4263 , piece_id=6HVCX7PTMB2Q4JGCFSG33NNPYWU52PJ2O52WJDQMEBU35L7442PA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769157858 (14h39m50.707573697s) [noise=true , address=45.91.116.12:4227 , piece_id=FCXFJQKSA2VBSOSSWQCIJAVOONBFJOLRTDCLOUPFUTSOSGUVPFGQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769433395 (14h39m50.707849244s) [noise=true , address=45.91.116.9:4262 , piece_id=ERT645SMMCLX4VBSL5NBGPVKAXQJFDQJVA7267TGZA73Y63DKPIA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769676039 (14h39m50.708091888s) [noise=true , address=45.91.116.7:4010 , piece_id=Y3PQKN2FEY63GQOHAH2NUEAW3YNLM2T465LOOGE35PBBM2G6IJ7A, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769945645 (14h39m50.708361494s) [noise=true , address=45.91.116.8:4047 , piece_id=ATA5VNNB3FY6JLM7XAC6NJAXP3XEMVAB6OJGA6J2RVKJDE7KSV6Q, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770185735 (14h39m50.708601574s) [noise=true , address=104.36.82.4:4803 , piece_id=P5YNTC577545WKRAFF5ZDVOZ7BKWIBW2YEPZNYS6R64V2SIEY52A, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770394053 (14h39m50.708809902s) [noise=true , address=45.91.116.4:4939 , piece_id=BHT5K3RVVCHUNCZ3NUYKPRUG43HT7AKE2AOBHLDDYL7SEVCXD3PQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770611088 (14h39m50.709026937s) [noise=true , address=77.90.31.16:4733 , piece_id=3ELJ3SXEPZFSIYAT72DLXB6W7K6PPBVL4ZY55INTYL7KH4KM2TEQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770800280 (14h39m50.709216129s) [noise=true , address=77.90.31.14:4663 , piece_id=6LDXGGVX3WY6S27SD3THJMFS7J3QAE4HD67L45N725KPD6B47KVQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770988741 (14h39m50.70940459s) [noise=true , address=104.36.82.4:4804 , piece_id=YC2KT2WSER5L3UORMDBVIFAIMCAFRPNBIAGAUTV2JT6YNXOE7YXA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.771221355 (14h39m50.709637204s) [noise=true , address=45.91.116.5:4298 , piece_id=F7LJFPUGTAV5WYMLIZPLPDTCUUDN3PIQGNRW4Y44MWTM4ZJHW53Q, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.771423853 (14h39m50.709839692s) [noise=true , address=104.36.82.16:4295 , piece_id=6C5W5RQSSRWFRUJE6GM5ODBIHTEOCQ33V473FLF5SVWYM62ARBRQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:44.484546105 (14h39m52.422961954s) [upload=14054]: Next returned nil segment Aug 7 04:48:44.484566966 (14h39m52.422982815s) [upload=14054]: Waiting for error group managing segments... Aug 7 04:48:44.782995940 (14h39m52.721411779s) [noise=true , address=45.91.116.7:4010 , piece_id=Y3PQKN2FEY63GQOHAH2NUEAW3YNLM2T465LOOGE35PBBM2G6IJ7A, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.786326542 (14h39m52.724742391s) [noise=true , address=45.91.116.7:4011 , piece_id=J3SAPWUNXY3VOJMT5PBRDCLTJIGQYHQLNN4RLFPLHH3P2UGTKGVA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.797538894 (14h39m52.735954743s) [noise=true , address=45.91.116.9:4262 , piece_id=ERT645SMMCLX4VBSL5NBGPVKAXQJFDQJVA7267TGZA73Y63DKPIA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.798511094 (14h39m52.736926943s) [noise=true , address=77.90.31.16:4733 , piece_id=3ELJ3SXEPZFSIYAT72DLXB6W7K6PPBVL4ZY55INTYL7KH4KM2TEQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.801571149 (14h39m52.739986988s) [noise=true , address=45.91.116.4:4939 , piece_id=BHT5K3RVVCHUNCZ3NUYKPRUG43HT7AKE2AOBHLDDYL7SEVCXD3PQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.815066949 (14h39m52.753482798s) [noise=true , address=104.36.82.4:4804 , piece_id=YC2KT2WSER5L3UORMDBVIFAIMCAFRPNBIAGAUTV2JT6YNXOE7YXA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.838459695 (14h39m52.776875544s) [noise=true , address=104.36.82.4:4803 , piece_id=P5YNTC577545WKRAFF5ZDVOZ7BKWIBW2YEPZNYS6R64V2SIEY52A, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.865483037 (14h39m52.803898886s) [noise=true , address=45.91.116.8:4047 , piece_id=ATA5VNNB3FY6JLM7XAC6NJAXP3XEMVAB6OJGA6J2RVKJDE7KSV6Q, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.867563558 (14h39m52.805979407s) [noise=true , address=45.91.116.6:4974 , piece_id=SAOVB5R5INZGVNDKOR5VL34FTPHKRO23PGX62ATGYEL5ZWHFVJBQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.872889537 (14h39m52.811305376s) [noise=true , address=45.91.116.12:4227 , piece_id=FCXFJQKSA2VBSOSSWQCIJAVOONBFJOLRTDCLOUPFUTSOSGUVPFGQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.891425040 (14h39m52.829840879s) [noise=true , address=104.36.82.8:4972 , piece_id=CZEHUAO5E55TW3S6YZVYJ2QPMUXUECKAOMYXZ6KW5EXEBWQDR2VA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.892049656 (14h39m52.830465495s) [noise=true , address=77.90.31.14:4663 , piece_id=6LDXGGVX3WY6S27SD3THJMFS7J3QAE4HD67L45N725KPD6B47KVQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.895282000 (14h39m52.833697849s) [noise=true , address=45.91.116.5:4298 , piece_id=F7LJFPUGTAV5WYMLIZPLPDTCUUDN3PIQGNRW4Y44MWTM4ZJHW53Q, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.895290786 (14h39m52.833706625s) [seg_pos={0 0}, upload=14054]: Segment reached optimal threshold of 13 pieces. Aug 7 04:48:44.895375949 (14h39m52.833791798s) [noise=true , address=45.91.116.9:4263 , piece_id=6HVCX7PTMB2Q4JGCFSG33NNPYWU52PJ2O52WJDQMEBU35L7442PA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895441084 (14h39m52.833856933s) [noise=true , address=104.36.82.16:4295 , piece_id=6C5W5RQSSRWFRUJE6GM5ODBIHTEOCQ33V473FLF5SVWYM62ARBRQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895506319 (14h39m52.833922168s) [noise=true , address=104.36.82.2:4730 , piece_id=LBXNDOJCZJTUO5JK4LIOXQULXBP4OYJ7U2EOHR6FAF25NVDPCCSQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895517911 (14h39m52.83393376s) [seg_pos={0 0}, upload=14054]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:48:44.895578357 (14h39m52.833994206s) [upload=14054]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:48:44.895582434 (14h39m52.833998273s) [upload=14054]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:48:45.212908141 (14h39m53.15132399s) [upload=14054]: Done uploading segments... Aug 7 04:48:45.212928370 (14h39m53.151344219s) [upload=14054]: Upload finished. err: <nil> Duration: 5.871024781s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................X..............................X...X....] duration: 5.926468224s FAIL (2025-08-07 04:51:53 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.926495637s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....X...............................X.......................] duration: 1.903025121s FAIL (2025-08-07 04:51:49 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.903058824s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................X..........X....] duration: 5.850461678s FAIL (2025-08-07 04:51:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== p3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:04.627096584 (14h39m12.565512433s) [noise=true , address=104.36.82.16:4293 , piece_id=RAUCGYLWLRML5WFEYIHTTLCQN2VACOSI6YVK7QILIJLEQZCZZZ4Q, seg_pos={0 0}, upload=14046]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:04.627136711 (14h39m12.56555256s) [seg_pos={0 0}, upload=14046]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:48:04.627218808 (14h39m12.565634658s) [upload=14046]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:48:04.627223087 (14h39m12.565638936s) [upload=14046]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:48:04.983981169 (14h39m12.922397018s) [upload=14046]: Done uploading segments... Aug 7 04:48:04.983999454 (14h39m12.922415303s) [upload=14046]: Upload finished. err: <nil> Aug 7 04:48:42.488465378 (14h39m50.426881227s) [upload=14054]: Starting upload Aug 7 04:48:42.488522797 (14h39m50.426938647s) [upload=14054]: Done starting upload Aug 7 04:48:42.488655972 (14h39m50.427071822s) [upload=14054]: Uploading segments... Aug 7 04:48:42.488722018 (14h39m50.427137857s) [upload=14054]: Got next segment. Inline: false Aug 7 04:48:42.488748830 (14h39m50.427164759s) [upload=14054]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:48:42.488753088 (14h39m50.427168937s) [upload=14054]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:48:42.768405779 (14h39m50.706821628s) [seg_pos={0 0}, upload=14054]: Begin upload segment... Aug 7 04:48:42.768450404 (14h39m50.706866253s) [seg_pos={0 0}, upload=14054]: Done begin upload segment. Aug 7 04:48:42.768463088 (14h39m50.706878927s) [noise=true , address=104.36.82.2:4730 , piece_id=LBXNDOJCZJTUO5JK4LIOXQULXBP4OYJ7U2EOHR6FAF25NVDPCCSQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768481714 (14h39m50.706897563s) [noise=true , address=45.91.116.7:4011 , piece_id=J3SAPWUNXY3VOJMT5PBRDCLTJIGQYHQLNN4RLFPLHH3P2UGTKGVA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768519626 (14h39m50.706935475s) [noise=true , address=45.91.116.6:4974 , piece_id=SAOVB5R5INZGVNDKOR5VL34FTPHKRO23PGX62ATGYEL5ZWHFVJBQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768824620 (14h39m50.707240469s) [noise=true , address=104.36.82.8:4972 , piece_id=CZEHUAO5E55TW3S6YZVYJ2QPMUXUECKAOMYXZ6KW5EXEBWQDR2VA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.768971191 (14h39m50.70738704s) [noise=true , address=45.91.116.9:4263 , piece_id=6HVCX7PTMB2Q4JGCFSG33NNPYWU52PJ2O52WJDQMEBU35L7442PA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769157858 (14h39m50.707573697s) [noise=true , address=45.91.116.12:4227 , piece_id=FCXFJQKSA2VBSOSSWQCIJAVOONBFJOLRTDCLOUPFUTSOSGUVPFGQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769433395 (14h39m50.707849244s) [noise=true , address=45.91.116.9:4262 , piece_id=ERT645SMMCLX4VBSL5NBGPVKAXQJFDQJVA7267TGZA73Y63DKPIA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769676039 (14h39m50.708091888s) [noise=true , address=45.91.116.7:4010 , piece_id=Y3PQKN2FEY63GQOHAH2NUEAW3YNLM2T465LOOGE35PBBM2G6IJ7A, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.769945645 (14h39m50.708361494s) [noise=true , address=45.91.116.8:4047 , piece_id=ATA5VNNB3FY6JLM7XAC6NJAXP3XEMVAB6OJGA6J2RVKJDE7KSV6Q, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770185735 (14h39m50.708601574s) [noise=true , address=104.36.82.4:4803 , piece_id=P5YNTC577545WKRAFF5ZDVOZ7BKWIBW2YEPZNYS6R64V2SIEY52A, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770394053 (14h39m50.708809902s) [noise=true , address=45.91.116.4:4939 , piece_id=BHT5K3RVVCHUNCZ3NUYKPRUG43HT7AKE2AOBHLDDYL7SEVCXD3PQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770611088 (14h39m50.709026937s) [noise=true , address=77.90.31.16:4733 , piece_id=3ELJ3SXEPZFSIYAT72DLXB6W7K6PPBVL4ZY55INTYL7KH4KM2TEQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770800280 (14h39m50.709216129s) [noise=true , address=77.90.31.14:4663 , piece_id=6LDXGGVX3WY6S27SD3THJMFS7J3QAE4HD67L45N725KPD6B47KVQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.770988741 (14h39m50.70940459s) [noise=true , address=104.36.82.4:4804 , piece_id=YC2KT2WSER5L3UORMDBVIFAIMCAFRPNBIAGAUTV2JT6YNXOE7YXA, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.771221355 (14h39m50.709637204s) [noise=true , address=45.91.116.5:4298 , piece_id=F7LJFPUGTAV5WYMLIZPLPDTCUUDN3PIQGNRW4Y44MWTM4ZJHW53Q, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:42.771423853 (14h39m50.709839692s) [noise=true , address=104.36.82.16:4295 , piece_id=6C5W5RQSSRWFRUJE6GM5ODBIHTEOCQ33V473FLF5SVWYM62ARBRQ, seg_pos={0 0}, upload=14054]: Uploading piece... Aug 7 04:48:44.484546105 (14h39m52.422961954s) [upload=14054]: Next returned nil segment Aug 7 04:48:44.484566966 (14h39m52.422982815s) [upload=14054]: Waiting for error group managing segments... Aug 7 04:48:44.782995940 (14h39m52.721411779s) [noise=true , address=45.91.116.7:4010 , piece_id=Y3PQKN2FEY63GQOHAH2NUEAW3YNLM2T465LOOGE35PBBM2G6IJ7A, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.786326542 (14h39m52.724742391s) [noise=true , address=45.91.116.7:4011 , piece_id=J3SAPWUNXY3VOJMT5PBRDCLTJIGQYHQLNN4RLFPLHH3P2UGTKGVA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.797538894 (14h39m52.735954743s) [noise=true , address=45.91.116.9:4262 , piece_id=ERT645SMMCLX4VBSL5NBGPVKAXQJFDQJVA7267TGZA73Y63DKPIA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.798511094 (14h39m52.736926943s) [noise=true , address=77.90.31.16:4733 , piece_id=3ELJ3SXEPZFSIYAT72DLXB6W7K6PPBVL4ZY55INTYL7KH4KM2TEQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.801571149 (14h39m52.739986988s) [noise=true , address=45.91.116.4:4939 , piece_id=BHT5K3RVVCHUNCZ3NUYKPRUG43HT7AKE2AOBHLDDYL7SEVCXD3PQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.815066949 (14h39m52.753482798s) [noise=true , address=104.36.82.4:4804 , piece_id=YC2KT2WSER5L3UORMDBVIFAIMCAFRPNBIAGAUTV2JT6YNXOE7YXA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.838459695 (14h39m52.776875544s) [noise=true , address=104.36.82.4:4803 , piece_id=P5YNTC577545WKRAFF5ZDVOZ7BKWIBW2YEPZNYS6R64V2SIEY52A, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.865483037 (14h39m52.803898886s) [noise=true , address=45.91.116.8:4047 , piece_id=ATA5VNNB3FY6JLM7XAC6NJAXP3XEMVAB6OJGA6J2RVKJDE7KSV6Q, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.867563558 (14h39m52.805979407s) [noise=true , address=45.91.116.6:4974 , piece_id=SAOVB5R5INZGVNDKOR5VL34FTPHKRO23PGX62ATGYEL5ZWHFVJBQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.872889537 (14h39m52.811305376s) [noise=true , address=45.91.116.12:4227 , piece_id=FCXFJQKSA2VBSOSSWQCIJAVOONBFJOLRTDCLOUPFUTSOSGUVPFGQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.891425040 (14h39m52.829840879s) [noise=true , address=104.36.82.8:4972 , piece_id=CZEHUAO5E55TW3S6YZVYJ2QPMUXUECKAOMYXZ6KW5EXEBWQDR2VA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.892049656 (14h39m52.830465495s) [noise=true , address=77.90.31.14:4663 , piece_id=6LDXGGVX3WY6S27SD3THJMFS7J3QAE4HD67L45N725KPD6B47KVQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.895282000 (14h39m52.833697849s) [noise=true , address=45.91.116.5:4298 , piece_id=F7LJFPUGTAV5WYMLIZPLPDTCUUDN3PIQGNRW4Y44MWTM4ZJHW53Q, seg_pos={0 0}, upload=14054]: Done uploading piece. err: <nil> Aug 7 04:48:44.895290786 (14h39m52.833706625s) [seg_pos={0 0}, upload=14054]: Segment reached optimal threshold of 13 pieces. Aug 7 04:48:44.895375949 (14h39m52.833791798s) [noise=true , address=45.91.116.9:4263 , piece_id=6HVCX7PTMB2Q4JGCFSG33NNPYWU52PJ2O52WJDQMEBU35L7442PA, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895441084 (14h39m52.833856933s) [noise=true , address=104.36.82.16:4295 , piece_id=6C5W5RQSSRWFRUJE6GM5ODBIHTEOCQ33V473FLF5SVWYM62ARBRQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895506319 (14h39m52.833922168s) [noise=true , address=104.36.82.2:4730 , piece_id=LBXNDOJCZJTUO5JK4LIOXQULXBP4OYJ7U2EOHR6FAF25NVDPCCSQ, seg_pos={0 0}, upload=14054]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom): protocol: expected piece hash; context canceled; EOF Aug 7 04:48:44.895517911 (14h39m52.83393376s) [seg_pos={0 0}, upload=14054]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:48:44.895578357 (14h39m52.833994206s) [upload=14054]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:48:44.895582434 (14h39m52.833998273s) [upload=14054]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:48:45.212908141 (14h39m53.15132399s) [upload=14054]: Done uploading segments... Aug 7 04:48:45.212928370 (14h39m53.151344219s) [upload=14054]: Upload finished. err: <nil> Duration: 5.850495333s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................X..............................X...X...] duration: 5.808708831s FAIL (2025-08-07 04:51:18 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.808742335s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.264638821s FAIL (2025-08-07 04:51:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.264662496s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.226646551s FAIL (2025-08-07 04:50:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.226669887s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....X...............................X......................] duration: 2.201051434s FAIL (2025-08-07 04:50:48 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 2.201081583s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)