Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X........X...X...........................................] duration: 5.690083214s FAIL (2025-08-08 18:39:11 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 5.690100917s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.8737903s FAIL (2025-08-08 18:38:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.873810689s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.....X....................X.....X...........X.X.............] duration: 1.826298786s FAIL (2025-08-08 18:38:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 18:29:29.638918462 (1h3m24.069998583s) [upload=1134]: Starting upload Aug 8 18:29:29.638968277 (1h3m24.070048408s) [upload=1134]: Done starting upload Aug 8 18:29:29.639028333 (1h3m24.070108464s) [upload=1134]: Uploading segments... Aug 8 18:29:29.639067086 (1h3m24.070147217s) [upload=1134]: Got next segment. Inline: true Aug 8 18:29:29.639070994 (1h3m24.070151125s) [upload=1134]: Waiting for error group managing segments... Aug 8 18:29:29.639182216 (1h3m24.070262338s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:29:29.639186725 (1h3m24.070266856s) [upload=1134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:29:29.639189180 (1h3m24.070269311s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:29:29.854169936 (1h3m24.285250067s) [upload=1134]: Done uploading segments... Aug 8 18:29:29.854198310 (1h3m24.285278441s) [upload=1134]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.826319775s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X........X...X..........................................] duration: 6.078679483s FAIL (2025-08-08 18:38:00 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 6.078705743s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [......X....................X.....X...........X.X............] duration: 2.017782689s FAIL (2025-08-08 18:37:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 18:29:29.638918462 (1h3m24.069998583s) [upload=1134]: Starting upload Aug 8 18:29:29.638968277 (1h3m24.070048408s) [upload=1134]: Done starting upload Aug 8 18:29:29.639028333 (1h3m24.070108464s) [upload=1134]: Uploading segments... Aug 8 18:29:29.639067086 (1h3m24.070147217s) [upload=1134]: Got next segment. Inline: true Aug 8 18:29:29.639070994 (1h3m24.070151125s) [upload=1134]: Waiting for error group managing segments... Aug 8 18:29:29.639182216 (1h3m24.070262338s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:29:29.639186725 (1h3m24.070266856s) [upload=1134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:29:29.639189180 (1h3m24.070269311s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:29:29.854169936 (1h3m24.285250067s) [upload=1134]: Done uploading segments... Aug 8 18:29:29.854198310 (1h3m24.285278441s) [upload=1134]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.017816123s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.......X....................X.....X...........X.X...........] duration: 1.850780838s FAIL (2025-08-08 18:37:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 18:29:29.638918462 (1h3m24.069998583s) [upload=1134]: Starting upload Aug 8 18:29:29.638968277 (1h3m24.070048408s) [upload=1134]: Done starting upload Aug 8 18:29:29.639028333 (1h3m24.070108464s) [upload=1134]: Uploading segments... Aug 8 18:29:29.639067086 (1h3m24.070147217s) [upload=1134]: Got next segment. Inline: true Aug 8 18:29:29.639070994 (1h3m24.070151125s) [upload=1134]: Waiting for error group managing segments... Aug 8 18:29:29.639182216 (1h3m24.070262338s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:29:29.639186725 (1h3m24.070266856s) [upload=1134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:29:29.639189180 (1h3m24.070269311s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:29:29.854169936 (1h3m24.285250067s) [upload=1134]: Done uploading segments... Aug 8 18:29:29.854198310 (1h3m24.285278441s) [upload=1134]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.850807939s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.168921511s FAIL (2025-08-08 18:37:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.168944835s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X........X...X.........................................] duration: 6.09179227s FAIL (2025-08-08 18:37:27 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 6.091818431s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X........X...X........................................] duration: 5.967421614s FAIL (2025-08-08 18:36:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 5.967451691s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.000172626s FAIL (2025-08-08 18:36:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.000191271s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [........X....................X.....X...........X.X..........] duration: 1.93959406s FAIL (2025-08-08 18:36:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 18:29:29.638918462 (1h3m24.069998583s) [upload=1134]: Starting upload Aug 8 18:29:29.638968277 (1h3m24.070048408s) [upload=1134]: Done starting upload Aug 8 18:29:29.639028333 (1h3m24.070108464s) [upload=1134]: Uploading segments... Aug 8 18:29:29.639067086 (1h3m24.070147217s) [upload=1134]: Got next segment. Inline: true Aug 8 18:29:29.639070994 (1h3m24.070151125s) [upload=1134]: Waiting for error group managing segments... Aug 8 18:29:29.639182216 (1h3m24.070262338s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:29:29.639186725 (1h3m24.070266856s) [upload=1134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:29:29.639189180 (1h3m24.070269311s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:29:29.854169936 (1h3m24.285250067s) [upload=1134]: Done uploading segments... Aug 8 18:29:29.854198310 (1h3m24.285278441s) [upload=1134]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.939619038s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X........X...X.......................................] duration: 5.568861691s FAIL (2025-08-08 18:35:39 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 5.568911969s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.374391246s FAIL (2025-08-08 18:35:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.374409931s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.........X....................X.....X...........X.X.........] duration: 1.869026655s FAIL (2025-08-08 18:35:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 18:29:29.638918462 (1h3m24.069998583s) [upload=1134]: Starting upload Aug 8 18:29:29.638968277 (1h3m24.070048408s) [upload=1134]: Done starting upload Aug 8 18:29:29.639028333 (1h3m24.070108464s) [upload=1134]: Uploading segments... Aug 8 18:29:29.639067086 (1h3m24.070147217s) [upload=1134]: Got next segment. Inline: true Aug 8 18:29:29.639070994 (1h3m24.070151125s) [upload=1134]: Waiting for error group managing segments... Aug 8 18:29:29.639182216 (1h3m24.070262338s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:29:29.639186725 (1h3m24.070266856s) [upload=1134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:29:29.639189180 (1h3m24.070269311s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:29:29.854169936 (1h3m24.285250067s) [upload=1134]: Done uploading segments... Aug 8 18:29:29.854198310 (1h3m24.285278441s) [upload=1134]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.869052716s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.589718489s FAIL (2025-08-08 18:35:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.589740582s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X........X...X......................................] duration: 6.113865573s FAIL (2025-08-08 18:34:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 6.113892815s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [..........X....................X.....X...........X.X........] duration: 2.014922958s FAIL (2025-08-08 18:34:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 18:29:29.638918462 (1h3m24.069998583s) [upload=1134]: Starting upload Aug 8 18:29:29.638968277 (1h3m24.070048408s) [upload=1134]: Done starting upload Aug 8 18:29:29.639028333 (1h3m24.070108464s) [upload=1134]: Uploading segments... Aug 8 18:29:29.639067086 (1h3m24.070147217s) [upload=1134]: Got next segment. Inline: true Aug 8 18:29:29.639070994 (1h3m24.070151125s) [upload=1134]: Waiting for error group managing segments... Aug 8 18:29:29.639182216 (1h3m24.070262338s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:29:29.639186725 (1h3m24.070266856s) [upload=1134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:29:29.639189180 (1h3m24.070269311s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:29:29.854169936 (1h3m24.285250067s) [upload=1134]: Done uploading segments... Aug 8 18:29:29.854198310 (1h3m24.285278441s) [upload=1134]: Upload finished. err: metaclient: Too Many Requests Duration:* 2.014946121s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X........X...X.....................................] duration: 6.245664184s FAIL (2025-08-08 18:34:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 6.24568858s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.011978834s FAIL (2025-08-08 18:34:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.011999463s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [X..........X....................X.....X...........X.X.......] duration: 1.992397939s FAIL (2025-08-08 18:34:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 18:29:29.638918462 (1h3m24.069998583s) [upload=1134]: Starting upload Aug 8 18:29:29.638968277 (1h3m24.070048408s) [upload=1134]: Done starting upload Aug 8 18:29:29.639028333 (1h3m24.070108464s) [upload=1134]: Uploading segments... Aug 8 18:29:29.639067086 (1h3m24.070147217s) [upload=1134]: Got next segment. Inline: true Aug 8 18:29:29.639070994 (1h3m24.070151125s) [upload=1134]: Waiting for error group managing segments... Aug 8 18:29:29.639182216 (1h3m24.070262338s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:29:29.639186725 (1h3m24.070266856s) [upload=1134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:29:29.639189180 (1h3m24.070269311s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:29:29.854169936 (1h3m24.285250067s) [upload=1134]: Done uploading segments... Aug 8 18:29:29.854198310 (1h3m24.285278441s) [upload=1134]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.992433446s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [.X..........X....................X.....X...........X.X......] duration: 1.86130427s FAIL (2025-08-08 18:33:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 18:29:29.638918462 (1h3m24.069998583s) [upload=1134]: Starting upload Aug 8 18:29:29.638968277 (1h3m24.070048408s) [upload=1134]: Done starting upload Aug 8 18:29:29.639028333 (1h3m24.070108464s) [upload=1134]: Uploading segments... Aug 8 18:29:29.639067086 (1h3m24.070147217s) [upload=1134]: Got next segment. Inline: true Aug 8 18:29:29.639070994 (1h3m24.070151125s) [upload=1134]: Waiting for error group managing segments... Aug 8 18:29:29.639182216 (1h3m24.070262338s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:29:29.639186725 (1h3m24.070266856s) [upload=1134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:29:29.639189180 (1h3m24.070269311s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:29:29.854169936 (1h3m24.285250067s) [upload=1134]: Done uploading segments... Aug 8 18:29:29.854198310 (1h3m24.285278441s) [upload=1134]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.861326542s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.452057962s FAIL (2025-08-08 18:33:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.452083832s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X........X...X....................................] duration: 5.891417518s FAIL (2025-08-08 18:33:09 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 5.891443377s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.057667848s FAIL (2025-08-08 18:32:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.057681524s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.926299555s FAIL (2025-08-08 18:32:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.926314454s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [..X..........X....................X.....X...........X.X.....] duration: 1.88115954s FAIL (2025-08-08 18:32:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 18:29:29.638918462 (1h3m24.069998583s) [upload=1134]: Starting upload Aug 8 18:29:29.638968277 (1h3m24.070048408s) [upload=1134]: Done starting upload Aug 8 18:29:29.639028333 (1h3m24.070108464s) [upload=1134]: Uploading segments... Aug 8 18:29:29.639067086 (1h3m24.070147217s) [upload=1134]: Got next segment. Inline: true Aug 8 18:29:29.639070994 (1h3m24.070151125s) [upload=1134]: Waiting for error group managing segments... Aug 8 18:29:29.639182216 (1h3m24.070262338s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:29:29.639186725 (1h3m24.070266856s) [upload=1134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:29:29.639189180 (1h3m24.070269311s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:29:29.854169936 (1h3m24.285250067s) [upload=1134]: Done uploading segments... Aug 8 18:29:29.854198310 (1h3m24.285278441s) [upload=1134]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.881185922s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X........X...X...................................] duration: 5.992891107s FAIL (2025-08-08 18:32:07 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 5.992917348s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X........X...X..................................] duration: 6.330748975s FAIL (2025-08-08 18:31:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 6.330774714s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [...X..........X....................X.....X...........X.X....] duration: 1.85926506s FAIL (2025-08-08 18:31:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 18:29:29.638918462 (1h3m24.069998583s) [upload=1134]: Starting upload Aug 8 18:29:29.638968277 (1h3m24.070048408s) [upload=1134]: Done starting upload Aug 8 18:29:29.639028333 (1h3m24.070108464s) [upload=1134]: Uploading segments... Aug 8 18:29:29.639067086 (1h3m24.070147217s) [upload=1134]: Got next segment. Inline: true Aug 8 18:29:29.639070994 (1h3m24.070151125s) [upload=1134]: Waiting for error group managing segments... Aug 8 18:29:29.639182216 (1h3m24.070262338s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:29:29.639186725 (1h3m24.070266856s) [upload=1134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:29:29.639189180 (1h3m24.070269311s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:29:29.854169936 (1h3m24.285250067s) [upload=1134]: Done uploading segments... Aug 8 18:29:29.854198310 (1h3m24.285278441s) [upload=1134]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.859284659s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.512168856s FAIL (2025-08-08 18:31:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.512185239s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X........X...X.................................] duration: 5.792606667s FAIL (2025-08-08 18:31:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 5.792633439s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [....X..........X....................X.....X...........X.X...] duration: 1.847623959s FAIL (2025-08-08 18:31:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 18:29:29.638918462 (1h3m24.069998583s) [upload=1134]: Starting upload Aug 8 18:29:29.638968277 (1h3m24.070048408s) [upload=1134]: Done starting upload Aug 8 18:29:29.639028333 (1h3m24.070108464s) [upload=1134]: Uploading segments... Aug 8 18:29:29.639067086 (1h3m24.070147217s) [upload=1134]: Got next segment. Inline: true Aug 8 18:29:29.639070994 (1h3m24.070151125s) [upload=1134]: Waiting for error group managing segments... Aug 8 18:29:29.639182216 (1h3m24.070262338s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:29:29.639186725 (1h3m24.070266856s) [upload=1134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:29:29.639189180 (1h3m24.070269311s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:29:29.854169936 (1h3m24.285250067s) [upload=1134]: Done uploading segments... Aug 8 18:29:29.854198310 (1h3m24.285278441s) [upload=1134]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.847646122s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [.....X..........X....................X.....X...........X.X..] duration: 1.896045164s FAIL (2025-08-08 18:30:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 18:29:29.638918462 (1h3m24.069998583s) [upload=1134]: Starting upload Aug 8 18:29:29.638968277 (1h3m24.070048408s) [upload=1134]: Done starting upload Aug 8 18:29:29.639028333 (1h3m24.070108464s) [upload=1134]: Uploading segments... Aug 8 18:29:29.639067086 (1h3m24.070147217s) [upload=1134]: Got next segment. Inline: true Aug 8 18:29:29.639070994 (1h3m24.070151125s) [upload=1134]: Waiting for error group managing segments... Aug 8 18:29:29.639182216 (1h3m24.070262338s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:29:29.639186725 (1h3m24.070266856s) [upload=1134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:29:29.639189180 (1h3m24.070269311s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:29:29.854169936 (1h3m24.285250067s) [upload=1134]: Done uploading segments... Aug 8 18:29:29.854198310 (1h3m24.285278441s) [upload=1134]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.89607862s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.295155316s FAIL (2025-08-08 18:30:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.29517847s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............X........X...X................................] duration: 6.138750324s FAIL (2025-08-08 18:30:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 6.138779821s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [......X..........X....................X.....X...........X.X.] duration: 1.871819993s FAIL (2025-08-08 18:30:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 18:29:29.638918462 (1h3m24.069998583s) [upload=1134]: Starting upload Aug 8 18:29:29.638968277 (1h3m24.070048408s) [upload=1134]: Done starting upload Aug 8 18:29:29.639028333 (1h3m24.070108464s) [upload=1134]: Uploading segments... Aug 8 18:29:29.639067086 (1h3m24.070147217s) [upload=1134]: Got next segment. Inline: true Aug 8 18:29:29.639070994 (1h3m24.070151125s) [upload=1134]: Waiting for error group managing segments... Aug 8 18:29:29.639182216 (1h3m24.070262338s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:29:29.639186725 (1h3m24.070266856s) [upload=1134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:29:29.639189180 (1h3m24.070269311s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:29:29.854169936 (1h3m24.285250067s) [upload=1134]: Done uploading segments... Aug 8 18:29:29.854198310 (1h3m24.285278441s) [upload=1134]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.871849067s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.233570142s FAIL (2025-08-08 18:29:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.233589378s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...............X........X...X...............................] duration: 5.760488742s FAIL (2025-08-08 18:29:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 5.760512888s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 6 out of 60 is failed (threshold=3) [.......X..........X....................X.....X...........X.X] duration: 865.415438ms FAIL (2025-08-08 18:29:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 18:29:29.638918462 (1h3m24.069998583s) [upload=1134]: Starting upload Aug 8 18:29:29.638968277 (1h3m24.070048408s) [upload=1134]: Done starting upload Aug 8 18:29:29.639028333 (1h3m24.070108464s) [upload=1134]: Uploading segments... Aug 8 18:29:29.639067086 (1h3m24.070147217s) [upload=1134]: Got next segment. Inline: true Aug 8 18:29:29.639070994 (1h3m24.070151125s) [upload=1134]: Waiting for error group managing segments... Aug 8 18:29:29.639182216 (1h3m24.070262338s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:29:29.639186725 (1h3m24.070266856s) [upload=1134]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:29:29.639189180 (1h3m24.070269311s) [upload=1134]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:29:29.854169936 (1h3m24.285250067s) [upload=1134]: Done uploading segments... Aug 8 18:29:29.854198310 (1h3m24.285278441s) [upload=1134]: Upload finished. err: metaclient: Too Many Requests Duration:* 865.435476ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [........X..........X....................X.....X...........X.] duration: 1.867663068s FAIL (2025-08-08 18:28:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 18:27:44.695464834 (1h1m39.126544965s) [upload=1101]: Starting upload Aug 8 18:27:44.695510662 (1h1m39.126590783s) [upload=1101]: Done starting upload Aug 8 18:27:44.695575506 (1h1m39.126655627s) [upload=1101]: Uploading segments... Aug 8 18:27:44.695609701 (1h1m39.126689832s) [upload=1101]: Got next segment. Inline: true Aug 8 18:27:44.695613388 (1h1m39.126693519s) [upload=1101]: Waiting for error group managing segments... Aug 8 18:27:44.695646982 (1h1m39.126727113s) [upload=1101]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:27:44.695650860 (1h1m39.126730991s) [upload=1101]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:27:44.695654647 (1h1m39.126734778s) [upload=1101]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:27:44.908569299 (1h1m39.33964944s) [upload=1101]: Done uploading segments... Aug 8 18:27:44.908635034 (1h1m39.339715165s) [upload=1101]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.86768005s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.233895187s FAIL (2025-08-08 18:28:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.233908171s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [................X........X...X..............................] duration: 5.951510425s FAIL (2025-08-08 18:28:15 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 5.95153433s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.134694053s FAIL (2025-08-08 18:27:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.134721345s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 5 out of 60 is failed (threshold=3) [.........X..........X....................X.....X...........X] duration: 861.471782ms FAIL (2025-08-08 18:27:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 8 18:27:44.695464834 (1h1m39.126544965s) [upload=1101]: Starting upload Aug 8 18:27:44.695510662 (1h1m39.126590783s) [upload=1101]: Done starting upload Aug 8 18:27:44.695575506 (1h1m39.126655627s) [upload=1101]: Uploading segments... Aug 8 18:27:44.695609701 (1h1m39.126689832s) [upload=1101]: Got next segment. Inline: true Aug 8 18:27:44.695613388 (1h1m39.126693519s) [upload=1101]: Waiting for error group managing segments... Aug 8 18:27:44.695646982 (1h1m39.126727113s) [upload=1101]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:27:44.695650860 (1h1m39.126730991s) [upload=1101]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:27:44.695654647 (1h1m39.126734778s) [upload=1101]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:27:44.908569299 (1h1m39.33964944s) [upload=1101]: Done uploading segments... Aug 8 18:27:44.908635034 (1h1m39.339715165s) [upload=1101]: Upload finished. err: metaclient: Too Many Requests Duration:* 861.492963ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.................X........X...X.............................] duration: 5.989047959s FAIL (2025-08-08 18:27:15 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 5.989070284s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..........X..........X....................X.....X...........] duration: 1.836918516s FAIL (2025-08-08 18:27:09 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 8 18:19:48.028749552 (53m42.459829683s) [upload=957]: Starting upload Aug 8 18:19:48.028798035 (53m42.459878156s) [upload=957]: Done starting upload Aug 8 18:19:48.028854182 (53m42.459934303s) [upload=957]: Uploading segments... Aug 8 18:19:48.028890873 (53m42.459971004s) [upload=957]: Got next segment. Inline: true Aug 8 18:19:48.028894790 (53m42.459974911s) [upload=957]: Waiting for error group managing segments... Aug 8 18:19:48.028934646 (53m42.460014777s) [upload=957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:19:48.028938643 (53m42.460018775s) [upload=957]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:19:48.028941188 (53m42.460021319s) [upload=957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:19:49.263618702 (53m43.694698833s) [upload=957]: Done uploading segments... Aug 8 18:19:49.263637899 (53m43.69471802s) [upload=957]: Upload finished. err: <nil> Duration:* 1.836952791s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.816007735s FAIL (2025-08-08 18:26:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.816044916s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..................X........X...X............................] duration: 5.805005766s FAIL (2025-08-08 18:26:47 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 5.805036283s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...........X..........X....................X.....X..........] duration: 1.863740786s FAIL (2025-08-08 18:26:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 8 18:19:48.028749552 (53m42.459829683s) [upload=957]: Starting upload Aug 8 18:19:48.028798035 (53m42.459878156s) [upload=957]: Done starting upload Aug 8 18:19:48.028854182 (53m42.459934303s) [upload=957]: Uploading segments... Aug 8 18:19:48.028890873 (53m42.459971004s) [upload=957]: Got next segment. Inline: true Aug 8 18:19:48.028894790 (53m42.459974911s) [upload=957]: Waiting for error group managing segments... Aug 8 18:19:48.028934646 (53m42.460014777s) [upload=957]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:19:48.028938643 (53m42.460018775s) [upload=957]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 8 18:19:48.028941188 (53m42.460021319s) [upload=957]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:19:49.263618702 (53m43.694698833s) [upload=957]: Done uploading segments... Aug 8 18:19:49.263637899 (53m43.69471802s) [upload=957]: Upload finished. err: <nil> Duration:* 1.863765514s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X........X...X...........................] duration: 5.877299846s FAIL (2025-08-08 18:26:06 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 8 18:08:32.158878721 (42m26.589958852s) [upload=750]: Starting upload Aug 8 18:08:32.158934127 (42m26.590014258s) [upload=750]: Done starting upload Aug 8 18:08:32.159014300 (42m26.590094431s) [upload=750]: Uploading segments... Aug 8 18:08:32.159154128 (42m26.590234329s) [upload=750]: Got next segment. Inline: false Aug 8 18:08:32.159193062 (42m26.590273183s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 8 18:08:32.159249370 (42m26.590329521s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 8 18:08:32.619210358 (42m27.050290489s) [seg_pos={0 0}, upload=750]: Begin upload segment... Aug 8 18:08:32.619249643 (42m27.050329774s) [seg_pos={0 0}, upload=750]: Done begin upload segment. Aug 8 18:08:32.619270152 (42m27.050350273s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.619823210 (42m27.050903341s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620150026 (42m27.051230147s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620523119 (42m27.05160325s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.620844254 (42m27.051924385s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621137575 (42m27.052217706s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621411269 (42m27.0524914s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621701694 (42m27.052781825s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.621960069 (42m27.0530402s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622234775 (42m27.053314906s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.622468722 (42m27.053548853s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623280185 (42m27.054360316s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.623774060 (42m27.054854181s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.624530928 (42m27.055611059s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.625404930 (42m27.056485061s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626153493 (42m27.057233624s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.626989051 (42m27.058069182s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:32.627383686 (42m27.058463817s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Uploading piece... Aug 8 18:08:33.730137164 (42m28.161217295s) [upload=750]: Next returned nil segment Aug 8 18:08:33.730151332 (42m28.161231463s) [upload=750]: Waiting for error group managing segments... Aug 8 18:08:34.014807988 (42m28.445888119s) [noise=true , address=79.127.205.233:4648 , piece_id=33KAEXCMQFSYG25BFZIGFNH5HANBOOTGIKL5KFKE7QSSO44JKIMQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.082739832 (42m28.513819963s) [noise=true , address=64.49.13.26:4948 , piece_id=LHQGTA554IREULAHP3Q3QSGY523MRWSGI45FPOBOW4PRGCJBPAMA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.104574428 (42m28.535654559s) [noise=true , address=64.49.13.154:4936 , piece_id=5JHATRFSB42SUVQG4LYG2UKJIVVZAPOYWRX2FAFEALXIWH4MWXDA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.560367250 (42m28.991447381s) [noise=true , address=104.36.82.60:4345 , piece_id=TX4PHOR2JBPYTIPEGZ7AIYZPBKCSCY2J3GVSEADUCT53WZBFVDQQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.573255500 (42m29.004335621s) [noise=true , address=104.36.82.60:4341 , piece_id=WTK3IBRT23CBOPSXSYUKRHGQSG622T2AUES376DCIKO66DSAOCUQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.642428328 (42m29.073508449s) [noise=true , address=104.36.82.59:4327 , piece_id=LF3HMRWMP4I26KPCKXVZ7T72B2WQVGZY7HM35VINRLGSPVSOJDNA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.649042744 (42m29.080122875s) [noise=true , address=104.36.82.35:4598 , piece_id=7JCMFN57EF2KRSJ4MGJOUILBK2DZABEVAHZMGDB2M6DKI3CHU7VQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.672062137 (42m29.103142268s) [noise=true , address=45.91.116.11:4129 , piece_id=BBCWG6ULHGBJKI42DQUAYTAG63VOM2UFV2KHCKUZH45TWCCDKA2A, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.683197252 (42m29.114277383s) [noise=true , address=104.36.82.46:4509 , piece_id=B46DOEJWSMG4M57P6VE3FT2EVG5G3VKJIC4YBGVVNBKMU3X2PGHQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.690802193 (42m29.121882314s) [noise=true , address=104.36.82.37:4654 , piece_id=MECYP7EZZFUPMAFEUFCVUE2WQTP5WRFOYTAKU6HUXHCX7IPNDMAQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.730638302 (42m29.161718433s) [noise=true , address=104.36.82.61:4379 , piece_id=3GYYNOMAMRBWYWEN23XBFOG5IZEFVFUEJWGPSXT6G2UADKIWE6WA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.756068527 (42m29.187148648s) [noise=true , address=104.36.82.52:4103 , piece_id=P6QV3H5NKFRSXXQKKVW3V5E2W6CQ6PGWITESTGATQYTLRNUHJNCQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.760324190 (42m29.191404321s) [noise=true , address=104.36.82.5:4858 , piece_id=KR2DROY56ECMCJ55VPXQSOOPVXJP6ZZCSI4BVPXLITAXB6PPKJBA, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770480252 (42m29.201560383s) [noise=true , address=104.36.82.15:4281 , piece_id=3SI2GJNEQ2PN5B76ZRNXAATOVLLRPETTQQXC53ZGPFDAIW7BO5XQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: <nil> Aug 8 18:08:34.770502485 (42m29.201582616s) [seg_pos={0 0}, upload=750]: Segment reached optimal threshold of 14 pieces. Aug 8 18:08:34.770595502 (42m29.201675623s) [noise=true , address=104.36.82.11:4143 , piece_id=OIYZCCHPFACRG7BCSOP2USITVTN6ZTY6JA5GI2R2WCBBWKOIQXPQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ipThd6PwZUpCXy3ML4qqMd9dmRLmxKjVNCNoT5wCg1szdWp8U): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770652110 (42m29.201732241s) [noise=true , address=104.36.82.44:4871 , piece_id=GQ57M3NGW7RHAEUVFICZJLXRY3D5LZOWSRZPUBE42RODCI565KXA, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12t3N84zSSMtzRBs1H47Szc2xAHwqmyr9szJahRkxMYDLto8DAx): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770818338 (42m29.201898469s) [noise=true , address=104.36.82.45:4929 , piece_id=ILJY377CZ5FBRVCFZCHZEMOLQZBM2LHQ67M7KOWHSXJQJOJTKKJQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VxQN6CV1ogwKcs27ho2ayknnPi7BDEcA9ZFbgsUacuQNFAdmm): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770822646 (42m29.201902778s) [noise=true , address=104.36.82.62:4416 , piece_id=UZECRJN7M6HC2Q5EOL5QEFBRHLVVTVS7NAU3E7HZSUEB434IXPKQ, seg_pos={0 0}, upload=750]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12syasUPsyYf4UstQ6cFAr6qamze3vCyAm9dnobk5e83w4bTz69): protocol: expected piece hash; context canceled; EOF Aug 8 18:08:34.770897630 (42m29.201977761s) [seg_pos={0 0}, upload=750]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 8 18:08:34.771018311 (42m29.202098442s) [upload=750]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 8 18:08:34.771025956 (42m29.202106127s) [upload=750]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 8 18:08:35.092714666 (42m29.523794797s) [upload=750]: Done uploading segments... Aug 8 18:08:35.092729535 (42m29.523809666s) [upload=750]: Upload finished. err: <nil> Duration: 5.877330384s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)