Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 1.929313664s FAIL (2025-08-07 05:10:29 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.92934273s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................X...X.....................X....] duration: 5.960476521s FAIL (2025-08-07 05:10:08 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.960505244s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X..........X..........X.....................] duration: 5.980624961s FAIL (2025-08-07 05:10:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.980652502s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................X...X.....................X...] duration: 5.71765013s FAIL (2025-08-07 05:09:48 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.717667915s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.228764498s FAIL (2025-08-07 05:09:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.228784838s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 2.24505749s FAIL (2025-08-07 05:09:33 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 2.245086585s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.340713117s FAIL (2025-08-07 05:09:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.340770929s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [X..............................X...X.....................X..] duration: 5.752992341s FAIL (2025-08-07 05:08:53 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.753020345s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X..........X..........X....................] duration: 5.953046167s FAIL (2025-08-07 05:08:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.953067539s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.996391449s FAIL (2025-08-07 05:08:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.996414403s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 1.828001389s FAIL (2025-08-07 05:08:37 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.828028549s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.92383759s FAIL (2025-08-07 05:08:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.923854513s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X..........X..........X...................] duration: 5.928275925s FAIL (2025-08-07 05:07:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.928300542s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 1.816472591s FAIL (2025-08-07 05:07:44 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.816500093s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X..............................X...X.....................X.] duration: 5.634177174s FAIL (2025-08-07 05:07:38 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.634201992s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.901641828s FAIL (2025-08-07 05:07:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.90166469s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X..........X..........X..................] duration: 5.894266416s FAIL (2025-08-07 05:07:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.894285772s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................X..........X..........X.................] duration: 5.814292337s FAIL (2025-08-07 05:06:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.814314822s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 2.185713967s FAIL (2025-08-07 05:06:33 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 2.185743093s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X..............................X...X.....................X] duration: 5.983158213s FAIL (2025-08-07 05:06:24 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.98318305s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.137303151s FAIL (2025-08-07 05:06:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.137324552s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................X..........X..........X................] duration: 5.933725249s FAIL (2025-08-07 05:06:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.933761069s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.939888782s FAIL (2025-08-07 05:05:59 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.939911265s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.210018276s FAIL (2025-08-07 05:05:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.210036702s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X..............................X...X.....................] duration: 5.61753746s FAIL (2025-08-07 05:05:37 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.617564822s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................X..........X..........X...............] duration: 5.847559353s FAIL (2025-08-07 05:05:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.847589711s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 1.927417228s FAIL (2025-08-07 05:05:18 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.927441664s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X..........X..........X..............] duration: 5.947704266s FAIL (2025-08-07 05:04:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.947730948s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.157876123s FAIL (2025-08-07 05:04:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.157899828s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X..............................X...X....................] duration: 5.772488044s FAIL (2025-08-07 05:04:33 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.772520026s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 1.914555582s FAIL (2025-08-07 05:04:20 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.914581751s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.312742402s FAIL (2025-08-07 05:04:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.312768322s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................X..........X..........X.............] duration: 5.787947146s FAIL (2025-08-07 05:04:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.787975689s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.174691223s FAIL (2025-08-07 05:03:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.174708087s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................X..........X..........X............] duration: 5.84908862s FAIL (2025-08-07 05:03:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.8491101s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X..............................X...X...................] duration: 5.557449796s FAIL (2025-08-07 05:03:30 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.55748878s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 1.90625096s FAIL (2025-08-07 05:03:09 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.906279183s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.009195911s FAIL (2025-08-07 05:02:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.009217191s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X..............................X...X..................] duration: 5.906283176s FAIL (2025-08-07 05:02:35 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.906343221s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................X..........X..........X...........] duration: 5.954986118s FAIL (2025-08-07 05:02:31 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.955015043s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.081017018s FAIL (2025-08-07 05:02:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.081034722s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 1.860645085s FAIL (2025-08-07 05:02:16 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.860672517s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X..............................X...X.................] duration: 5.642554313s FAIL (2025-08-07 05:02:11 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.642576606s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X..............................X...X................] duration: 5.774577537s FAIL (2025-08-07 05:01:54 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.774607174s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.0625786s FAIL (2025-08-07 05:01:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.062599401s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................X..........X..........X..........] duration: 6.616514729s FAIL (2025-08-07 05:01:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 6.616555126s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.314270826s FAIL (2025-08-07 05:01:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.314293689s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 2.099093299s FAIL (2025-08-07 05:01:03 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 2.099133206s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................X..........X..........X.........] duration: 6.012381506s FAIL (2025-08-07 05:00:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 6.012411704s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X..............................X...X...............] duration: 5.643210003s FAIL (2025-08-07 05:00:55 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 5a3a14d953f6107f998203c3ab90c3530e38c5175aed488f3b20a5db0e9dfcb7 != before 9dfbe2011caf7f350097023a753d10c10ba97747007712d2675c31d895fcd823 Duration: 5.643233176s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)