Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X..........X.......................................] duration: 5.773207524s FAIL (2025-08-07 05:23:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.773242499s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X..........X......................................] duration: 5.897068997s FAIL (2025-08-07 05:23:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.897096079s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X...X.....................X.....................] duration: 5.77191116s FAIL (2025-08-07 05:23:19 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.771940886s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X...X.....................X....................] duration: 5.83924281s FAIL (2025-08-07 05:23:03 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.839271735s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.98761938s FAIL (2025-08-07 05:22:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.987644348s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [X..........X..........X.....................................] duration: 5.807177294s FAIL (2025-08-07 05:22:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.807202544s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X...X.....................X...................] duration: 5.699387645s FAIL (2025-08-07 05:22:23 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.699412984s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X..........X..........X....................................] duration: 5.769954657s FAIL (2025-08-07 05:22:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.769990083s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.383233307s FAIL (2025-08-07 05:21:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.38325515s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..X..........X..........X...................................] duration: 5.76806469s FAIL (2025-08-07 05:21:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.768091792s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X...X.....................X..................] duration: 5.605044237s FAIL (2025-08-07 05:21:30 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.605063225s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X...X.....................X.................] duration: 5.658819717s FAIL (2025-08-07 05:20:50 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.658839574s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.504495067s FAIL (2025-08-07 05:20:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.5045184s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X..........X..........X..................................] duration: 5.879666697s FAIL (2025-08-07 05:20:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.879695173s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.956446667s FAIL (2025-08-07 05:20:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.956461384s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X..........X..........X.................................] duration: 5.762745834s FAIL (2025-08-07 05:20:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.762769558s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X...X.....................X................] duration: 5.778697133s FAIL (2025-08-07 05:19:43 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.778725046s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X..........X..........X................................] duration: 5.928876585s FAIL (2025-08-07 05:19:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.928901963s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.226853406s FAIL (2025-08-07 05:19:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.226875347s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.851036671s FAIL (2025-08-07 05:19:08 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.851071808s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X...X.....................X...............] duration: 5.669274841s FAIL (2025-08-07 05:19:04 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.669304497s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.121067291s FAIL (2025-08-07 05:18:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.121088993s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X..........X..........X...............................] duration: 5.718229861s FAIL (2025-08-07 05:18:32 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.718253237s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X...X.....................X..............] duration: 5.787014026s FAIL (2025-08-07 05:18:18 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.78703738s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.164944464s FAIL (2025-08-07 05:18:07 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 2.164967038s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.098834843s FAIL (2025-08-07 05:17:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.098855443s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X..........X..........X..............................] duration: 5.747937782s FAIL (2025-08-07 05:17:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.747969032s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................X...X.....................X.............] duration: 5.871483677s FAIL (2025-08-07 05:17:39 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.871505689s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.080398403s FAIL (2025-08-07 05:17:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.080419083s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.040326805s FAIL (2025-08-07 05:17:32 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 2.040353034s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................X...X.....................X............] duration: 5.583860619s FAIL (2025-08-07 05:17:11 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.583892831s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.85705732s FAIL (2025-08-07 05:17:10 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.857078372s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.7471821s FAIL (2025-08-07 05:16:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.747205735s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X..........X..........X.............................] duration: 5.799865656s FAIL (2025-08-07 05:16:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.799897386s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.887543658s FAIL (2025-08-07 05:16:35 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.887576421s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................X...X.....................X...........] duration: 5.655056132s FAIL (2025-08-07 05:16:27 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.655076843s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.959377224s FAIL (2025-08-07 05:16:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.959404495s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X..........X..........X............................] duration: 5.714385856s FAIL (2025-08-07 05:16:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.714413229s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................X...X.....................X..........] duration: 5.690138757s FAIL (2025-08-07 05:16:03 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.690158726s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 1.836779061s FAIL (2025-08-07 05:15:47 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.836803709s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.406201533s FAIL (2025-08-07 05:15:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.40622074s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.055844662s FAIL (2025-08-07 05:15:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.055866663s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X..........X..........X...........................] duration: 5.792001357s FAIL (2025-08-07 05:15:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.792026245s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................X...X.....................X.........] duration: 5.973613583s FAIL (2025-08-07 05:14:54 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.973641376s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.976467551s FAIL (2025-08-07 05:14:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.976495716s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.818269168s FAIL (2025-08-07 05:14:34 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 1.818294675s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X..........X..........X..........................] duration: 5.821282771s FAIL (2025-08-07 05:14:10 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476517671 (14h45m50.41493352s) [noise=true , address=104.36.82.6:4874 , piece_id=ZFD6NPNAAUVIRZ5YGTKCBJPLQXPFJMN25HQT7FUNU26DPTS2F7UQ, seg_pos={0 0}, upload=14153]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71): protocol: expected piece hash; context canceled; EOF Aug 7 04:54:42.476561945 (14h45m50.414977794s) [seg_pos={0 0}, upload=14153]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:54:42.476635777 (14h45m50.415051626s) [upload=14153]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:54:42.476640095 (14h45m50.415055944s) [upload=14153]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:54:42.859043030 (14h45m50.797458879s) [upload=14153]: Done uploading segments... Aug 7 04:54:42.859063249 (14h45m50.797479098s) [upload=14153]: Upload finished. err: <nil> Aug 7 04:55:05.836870332 (14h46m13.775286181s) [upload=14159]: Starting upload Aug 7 04:55:05.836926720 (14h46m13.775342569s) [upload=14159]: Done starting upload Aug 7 04:55:05.837015199 (14h46m13.775431048s) [upload=14159]: Uploading segments... Aug 7 04:55:05.837116423 (14h46m13.775532272s) [upload=14159]: Got next segment. Inline: false Aug 7 04:55:05.837155838 (14h46m13.775571687s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 7 04:55:05.837160236 (14h46m13.775576086s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 7 04:55:06.073286783 (14h46m14.011702622s) [seg_pos={0 0}, upload=14159]: Begin upload segment... Aug 7 04:55:06.073338001 (14h46m14.01175389s) [seg_pos={0 0}, upload=14159]: Done begin upload segment. Aug 7 04:55:06.073380742 (14h46m14.011796651s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073390892 (14h46m14.011806741s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073867794 (14h46m14.012283643s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.073885418 (14h46m14.012301267s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074211331 (14h46m14.01262718s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074520083 (14h46m14.012935922s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.074805158 (14h46m14.013221017s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075051048 (14h46m14.013466897s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075279705 (14h46m14.013695554s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075364017 (14h46m14.013779866s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075645796 (14h46m14.014061645s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.075834126 (14h46m14.014249975s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076062343 (14h46m14.014478192s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076228621 (14h46m14.01464447s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076500751 (14h46m14.0149166s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:06.076692338 (14h46m14.015108187s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Uploading piece... Aug 7 04:55:07.706338803 (14h46m15.644754653s) [upload=14159]: Next returned nil segment Aug 7 04:55:07.706356186 (14h46m15.644772035s) [upload=14159]: Waiting for error group managing segments... Aug 7 04:55:08.079467986 (14h46m16.017883835s) [noise=true , address=45.91.116.7:4009 , piece_id=C2MHRJ4CI4AYL74TJ4KOBIQC3FAXGG742O62MSRSMFEKGJ2JS3HA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.083077803 (14h46m16.021493662s) [noise=true , address=45.91.116.8:4045 , piece_id=ZVGXB673GLM2FMZFVJXUCTQL2S7INOLAIUBRFSEAX4ZBBWUN7H3Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.086948248 (14h46m16.025364097s) [noise=true , address=77.90.31.15:4697 , piece_id=J2KPIRWKL7B4SOZNCH2EOD5FQCR3C5JQBIIVVYWAEKRNDAE3AX4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.096855863 (14h46m16.035271702s) [noise=true , address=77.90.31.12:4592 , piece_id=C5KKUD24FGEDIJO3RTD5PAEF5GETZR2BYTDZ66ZSPZRCRLY34UCA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.099582620 (14h46m16.037998469s) [noise=true , address=45.91.116.8:4047 , piece_id=WEFN62SUQLYB7AFZSZQ66RDHX2MSEPUSKY33YPOPLQAXRZU7SHWA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.103264886 (14h46m16.041680735s) [noise=true , address=77.90.31.14:4662 , piece_id=OI2J6BBDKFJ7G2UYY3QQ74IXBPQUDBG5RYWX3ZVGMRNXBGXAY5LQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112037449 (14h46m16.050453298s) [noise=true , address=104.36.82.13:4187 , piece_id=DEIA5S3YXGTHCCD7P2BC6KXNXZ6SALAC4NUVRNQF7RYKY3GWTZ3A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.112072888 (14h46m16.050488727s) [noise=true , address=45.91.116.10:4081 , piece_id=55YNSM2WVZBI3VO45RS6PKHH57UTGM32SSO2QNL5N2MBTV3OTJZQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.140507229 (14h46m16.078923078s) [noise=true , address=104.36.82.16:4295 , piece_id=ZPEDB2LZA6M7TXOZFLZ2MCRBHVDUZ4FMTSJVOCYSJVC7CIIJB57Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.178535946 (14h46m16.116951795s) [noise=true , address=45.91.116.11:4118 , piece_id=JCSO3RMSRG6JYFN63ZAWFKQ6GBG25QHCACMED6GF5KF2YSIPAB4Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.181286579 (14h46m16.119702428s) [noise=true , address=45.91.116.12:4225 , piece_id=A4KTBBXML6OW66FAQLEJ3PDFZHWOEY2KCJQBT6WLVOZDDWHKQCGA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.186892866 (14h46m16.125308715s) [noise=true , address=45.91.116.13:4155 , piece_id=TTIPSV6C43LMKAJVFGESR55JRF5T5HDKO4VQLQJJ7G3ENPVQMMPA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193444219 (14h46m16.131860078s) [noise=true , address=104.36.82.12:4151 , piece_id=XPOKTPIDNSTZL2KJRMMEDFNUHNDYGXUGRRWFDJDYVMX3CLIU4EXQ, seg_pos={0 0}, upload=14159]: Done uploading piece. err: <nil> Aug 7 04:55:08.193468486 (14h46m16.131884335s) [seg_pos={0 0}, upload=14159]: Segment reached optimal threshold of 13 pieces. Aug 7 04:55:08.193605848 (14h46m16.132021697s) [noise=true , address=104.36.82.2:4732 , piece_id=D6TWGHGA5Q2VZJUWHR2JH7LKM6NJ6CCAPBI2YUJWDJJ5LTGTGX7Q, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193660403 (14h46m16.132076252s) [noise=true , address=77.90.31.14:4663 , piece_id=JMUAHCSTIKPLBNU6GBBKH5K75OSDCUJTZAXK4RBL6SGDFTSSDNMA, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193695380 (14h46m16.132111229s) [noise=true , address=104.36.82.11:4114 , piece_id=XGLX4HD6BRRZUJOJUZZE7XF32NENFHT4HMBQAIXCNTPQMKFNDZ2A, seg_pos={0 0}, upload=14159]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN): protocol: expected piece hash; context canceled; EOF Aug 7 04:55:08.193712683 (14h46m16.132128532s) [seg_pos={0 0}, upload=14159]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 7 04:55:08.193822934 (14h46m16.132238783s) [upload=14159]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 7 04:55:08.193827452 (14h46m16.132243301s) [upload=14159]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 7 04:55:08.487892748 (14h46m16.426308607s) [upload=14159]: Done uploading segments... Aug 7 04:55:08.487913047 (14h46m16.426328896s) [upload=14159]: Upload finished. err: <nil> Duration: 5.82131393s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.037781475s FAIL (2025-08-07 05:13:58 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/1K):: after 11c322f7b1eb2f7e8292e4637fb6451967e8193eeab2f4a37db3ef25b5465093 != before 91e3712b8bb140f6dc7e3b050637abeb30e614ca7e6c42795cd061535652c0ed Duration: 2.037816802s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................X...X.....................X........] duration: 5.681699948s FAIL (2025-08-07 05:13:41 UTC) orbiter warning: operation: sync2.Copy digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 2a1b3ec28e7199156a3f4d4616d0d5eed5ab214532aeb49ecb9757cbb62da413 != before 57bd9df9aae0e0164ea9509dedb90a3f1d2caf9fb2f66475a29f29b5ff5ef938 Duration: 5.68172659s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.939973948s FAIL (2025-08-07 05:13:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.93999665s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)