Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [X....X.........................X......X.....XXX........X....] duration: 6.248654419s FAIL (2025-08-09 04:17:22 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.248679907s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [................X................................X..........] duration: 2.53839696s FAIL (2025-08-09 04:17:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.538415946s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................................X........X...............] duration: 1.835520323s FAIL (2025-08-09 04:17:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 04:07:10.334620288 (10h41m4.765700419s) [upload=11163]: Starting upload Aug 9 04:07:10.334666016 (10h41m4.765746147s) [upload=11163]: Done starting upload Aug 9 04:07:10.334726181 (10h41m4.765806312s) [upload=11163]: Uploading segments... Aug 9 04:07:10.334759073 (10h41m4.765839204s) [upload=11163]: Got next segment. Inline: true Aug 9 04:07:10.334763161 (10h41m4.765843292s) [upload=11163]: Waiting for error group managing segments... Aug 9 04:07:10.334796215 (10h41m4.765876336s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:07:10.334800333 (10h41m4.765880464s) [upload=11163]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 04:07:10.334802918 (10h41m4.765883049s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:07:11.359905316 (10h41m5.790985467s) [upload=11163]: Done uploading segments... Aug 9 04:07:11.359924452 (10h41m5.791004594s) [upload=11163]: Upload finished. err: <nil> Duration:* 1.835547906s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X............X.............................X............] duration: 5.885353529s FAIL (2025-08-09 04:17:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 523f30c4aaf94ddc9fbf8b4aeb9a36d20987df7a771e62b662b689d1413f1fbc != before 825f4c11ac74d2704e1d6c6d1b139ad0fcebca64bd571c7b27bbf33fb59231d4 ======LOGS====== Aug 9 04:09:40.375954558 (10h43m34.807034749s) [upload=11204]: Starting upload Aug 9 04:09:40.376058546 (10h43m34.807138677s) [upload=11204]: Done starting upload Aug 9 04:09:40.376218022 (10h43m34.807298153s) [upload=11204]: Uploading segments... Aug 9 04:09:40.376312062 (10h43m34.807392193s) [upload=11204]: Got next segment. Inline: false Aug 9 04:09:40.376344544 (10h43m34.807424675s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:09:40.376349013 (10h43m34.807429144s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 04:09:40.873631448 (10h43m35.304711579s) [seg_pos={0 0}, upload=11204]: Begin upload segment... Aug 9 04:09:40.873691684 (10h43m35.304771815s) [seg_pos={0 0}, upload=11204]: Done begin upload segment. Aug 9 04:09:40.873713004 (10h43m35.304793135s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.873755876 (10h43m35.304836007s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874170830 (10h43m35.305250951s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874868184 (10h43m35.305948315s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875229585 (10h43m35.306309706s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875522877 (10h43m35.306603008s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.877073413 (10h43m35.308153544s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.878880160 (10h43m35.309960281s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879375929 (10h43m35.31045605s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879497672 (10h43m35.310577803s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879783989 (10h43m35.310864121s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879811903 (10h43m35.310892034s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880014260 (10h43m35.311094391s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880236695 (10h43m35.311316826s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880492004 (10h43m35.311572135s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880711344 (10h43m35.311791465s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881026116 (10h43m35.312106247s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881320319 (10h43m35.31240045s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:42.031531447 (10h43m36.462611578s) [upload=11204]: Next returned nil segment Aug 9 04:09:42.031550193 (10h43m36.462630324s) [upload=11204]: Waiting for error group managing segments... Aug 9 04:09:42.321166913 (10h43m36.752247034s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.777697238 (10h43m37.208777369s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.789954881 (10h43m37.221035002s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.790550711 (10h43m37.221630842s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.812786204 (10h43m37.243866335s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.814899077 (10h43m37.245979208s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.823123292 (10h43m37.254203423s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.828425747 (10h43m37.259505878s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.834875747 (10h43m37.265955879s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.835243621 (10h43m37.266323742s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.874904975 (10h43m37.305985106s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.889634908 (10h43m37.320715039s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.890729572 (10h43m37.321809703s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913109842 (10h43m37.344189973s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913131072 (10h43m37.344211193s) [seg_pos={0 0}, upload=11204]: Segment reached optimal threshold of 14 pieces. Aug 9 04:09:42.913224241 (10h43m37.344304372s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sF7TCu1pL8mrtjoPULJ2VnZQHcJSMmvou2w7YgF58ZHQcnfJT): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913270629 (10h43m37.34435075s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16aJgn61bhumVFmXfTv8YUi5JfZedtBqadv8yVcSEjgfgUxuDg): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913300577 (10h43m37.344380708s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xZEzok8AN2SUW6wups5Qug2TndRgrwkK2XMFvXfRyZcXFZRbU): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913331346 (10h43m37.344411477s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913347877 (10h43m37.344428008s) [seg_pos={0 0}, upload=11204]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 04:09:42.913400328 (10h43m37.344480459s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 04:09:42.913404055 (10h43m37.344484176s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:09:43.578329530 (10h43m38.009409661s) [upload=11204]: Done uploading segments... Aug 9 04:09:43.578350230 (10h43m38.009430361s) [upload=11204]: Upload finished. err: <nil> Duration: 5.885382806s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 778.87321ms FAIL (2025-08-09 04:17:07 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 778.893198ms edge authservice http us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 2m0.000382143s FAIL (2025-08-09 04:17:06 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 2m0.00040749s edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X............X.............................X...........] duration: 6.187415447s FAIL (2025-08-09 04:17:00 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 523f30c4aaf94ddc9fbf8b4aeb9a36d20987df7a771e62b662b689d1413f1fbc != before 825f4c11ac74d2704e1d6c6d1b139ad0fcebca64bd571c7b27bbf33fb59231d4 ======LOGS====== Aug 9 04:09:40.375954558 (10h43m34.807034749s) [upload=11204]: Starting upload Aug 9 04:09:40.376058546 (10h43m34.807138677s) [upload=11204]: Done starting upload Aug 9 04:09:40.376218022 (10h43m34.807298153s) [upload=11204]: Uploading segments... Aug 9 04:09:40.376312062 (10h43m34.807392193s) [upload=11204]: Got next segment. Inline: false Aug 9 04:09:40.376344544 (10h43m34.807424675s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:09:40.376349013 (10h43m34.807429144s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 04:09:40.873631448 (10h43m35.304711579s) [seg_pos={0 0}, upload=11204]: Begin upload segment... Aug 9 04:09:40.873691684 (10h43m35.304771815s) [seg_pos={0 0}, upload=11204]: Done begin upload segment. Aug 9 04:09:40.873713004 (10h43m35.304793135s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.873755876 (10h43m35.304836007s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874170830 (10h43m35.305250951s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874868184 (10h43m35.305948315s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875229585 (10h43m35.306309706s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875522877 (10h43m35.306603008s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.877073413 (10h43m35.308153544s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.878880160 (10h43m35.309960281s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879375929 (10h43m35.31045605s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879497672 (10h43m35.310577803s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879783989 (10h43m35.310864121s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879811903 (10h43m35.310892034s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880014260 (10h43m35.311094391s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880236695 (10h43m35.311316826s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880492004 (10h43m35.311572135s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880711344 (10h43m35.311791465s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881026116 (10h43m35.312106247s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881320319 (10h43m35.31240045s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:42.031531447 (10h43m36.462611578s) [upload=11204]: Next returned nil segment Aug 9 04:09:42.031550193 (10h43m36.462630324s) [upload=11204]: Waiting for error group managing segments... Aug 9 04:09:42.321166913 (10h43m36.752247034s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.777697238 (10h43m37.208777369s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.789954881 (10h43m37.221035002s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.790550711 (10h43m37.221630842s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.812786204 (10h43m37.243866335s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.814899077 (10h43m37.245979208s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.823123292 (10h43m37.254203423s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.828425747 (10h43m37.259505878s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.834875747 (10h43m37.265955879s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.835243621 (10h43m37.266323742s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.874904975 (10h43m37.305985106s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.889634908 (10h43m37.320715039s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.890729572 (10h43m37.321809703s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913109842 (10h43m37.344189973s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913131072 (10h43m37.344211193s) [seg_pos={0 0}, upload=11204]: Segment reached optimal threshold of 14 pieces. Aug 9 04:09:42.913224241 (10h43m37.344304372s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sF7TCu1pL8mrtjoPULJ2VnZQHcJSMmvou2w7YgF58ZHQcnfJT): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913270629 (10h43m37.34435075s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16aJgn61bhumVFmXfTv8YUi5JfZedtBqadv8yVcSEjgfgUxuDg): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913300577 (10h43m37.344380708s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xZEzok8AN2SUW6wups5Qug2TndRgrwkK2XMFvXfRyZcXFZRbU): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913331346 (10h43m37.344411477s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913347877 (10h43m37.344428008s) [seg_pos={0 0}, upload=11204]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 04:09:42.913400328 (10h43m37.344480459s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 04:09:42.913404055 (10h43m37.344484176s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:09:43.578329530 (10h43m38.009409661s) [upload=11204]: Done uploading segments... Aug 9 04:09:43.578350230 (10h43m38.009430361s) [upload=11204]: Upload finished. err: <nil> Duration: 6.187446124s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [....................................X........X..............] duration: 1.875092924s FAIL (2025-08-09 04:16:46 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 04:07:10.334620288 (10h41m4.765700419s) [upload=11163]: Starting upload Aug 9 04:07:10.334666016 (10h41m4.765746147s) [upload=11163]: Done starting upload Aug 9 04:07:10.334726181 (10h41m4.765806312s) [upload=11163]: Uploading segments... Aug 9 04:07:10.334759073 (10h41m4.765839204s) [upload=11163]: Got next segment. Inline: true Aug 9 04:07:10.334763161 (10h41m4.765843292s) [upload=11163]: Waiting for error group managing segments... Aug 9 04:07:10.334796215 (10h41m4.765876336s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:07:10.334800333 (10h41m4.765880464s) [upload=11163]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 04:07:10.334802918 (10h41m4.765883049s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:07:11.359905316 (10h41m5.790985467s) [upload=11163]: Done uploading segments... Aug 9 04:07:11.359924452 (10h41m5.791004594s) [upload=11163]: Upload finished. err: <nil> Duration:* 1.875118693s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.X....X.........................X......X.....XXX........X...] duration: 8.626736699s FAIL (2025-08-09 04:16:33 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 8.626750495s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X........X.............] duration: 2.15286788s FAIL (2025-08-09 04:16:26 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 04:07:10.334620288 (10h41m4.765700419s) [upload=11163]: Starting upload Aug 9 04:07:10.334666016 (10h41m4.765746147s) [upload=11163]: Done starting upload Aug 9 04:07:10.334726181 (10h41m4.765806312s) [upload=11163]: Uploading segments... Aug 9 04:07:10.334759073 (10h41m4.765839204s) [upload=11163]: Got next segment. Inline: true Aug 9 04:07:10.334763161 (10h41m4.765843292s) [upload=11163]: Waiting for error group managing segments... Aug 9 04:07:10.334796215 (10h41m4.765876336s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:07:10.334800333 (10h41m4.765880464s) [upload=11163]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 04:07:10.334802918 (10h41m4.765883049s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:07:11.359905316 (10h41m5.790985467s) [upload=11163]: Done uploading segments... Aug 9 04:07:11.359924452 (10h41m5.791004594s) [upload=11163]: Upload finished. err: <nil> Duration:* 2.152898238s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.................X................................X.........] duration: 2.734988706s FAIL (2025-08-09 04:16:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.735014336s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..X....X.........................X......X.....XXX........X..] duration: 6.342605764s FAIL (2025-08-09 04:16:09 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.342631225s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X............X.............................X..........] duration: 5.831629189s FAIL (2025-08-09 04:16:02 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 523f30c4aaf94ddc9fbf8b4aeb9a36d20987df7a771e62b662b689d1413f1fbc != before 825f4c11ac74d2704e1d6c6d1b139ad0fcebca64bd571c7b27bbf33fb59231d4 ======LOGS====== Aug 9 04:09:40.375954558 (10h43m34.807034749s) [upload=11204]: Starting upload Aug 9 04:09:40.376058546 (10h43m34.807138677s) [upload=11204]: Done starting upload Aug 9 04:09:40.376218022 (10h43m34.807298153s) [upload=11204]: Uploading segments... Aug 9 04:09:40.376312062 (10h43m34.807392193s) [upload=11204]: Got next segment. Inline: false Aug 9 04:09:40.376344544 (10h43m34.807424675s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:09:40.376349013 (10h43m34.807429144s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 04:09:40.873631448 (10h43m35.304711579s) [seg_pos={0 0}, upload=11204]: Begin upload segment... Aug 9 04:09:40.873691684 (10h43m35.304771815s) [seg_pos={0 0}, upload=11204]: Done begin upload segment. Aug 9 04:09:40.873713004 (10h43m35.304793135s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.873755876 (10h43m35.304836007s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874170830 (10h43m35.305250951s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874868184 (10h43m35.305948315s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875229585 (10h43m35.306309706s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875522877 (10h43m35.306603008s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.877073413 (10h43m35.308153544s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.878880160 (10h43m35.309960281s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879375929 (10h43m35.31045605s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879497672 (10h43m35.310577803s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879783989 (10h43m35.310864121s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879811903 (10h43m35.310892034s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880014260 (10h43m35.311094391s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880236695 (10h43m35.311316826s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880492004 (10h43m35.311572135s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880711344 (10h43m35.311791465s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881026116 (10h43m35.312106247s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881320319 (10h43m35.31240045s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:42.031531447 (10h43m36.462611578s) [upload=11204]: Next returned nil segment Aug 9 04:09:42.031550193 (10h43m36.462630324s) [upload=11204]: Waiting for error group managing segments... Aug 9 04:09:42.321166913 (10h43m36.752247034s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.777697238 (10h43m37.208777369s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.789954881 (10h43m37.221035002s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.790550711 (10h43m37.221630842s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.812786204 (10h43m37.243866335s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.814899077 (10h43m37.245979208s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.823123292 (10h43m37.254203423s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.828425747 (10h43m37.259505878s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.834875747 (10h43m37.265955879s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.835243621 (10h43m37.266323742s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.874904975 (10h43m37.305985106s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.889634908 (10h43m37.320715039s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.890729572 (10h43m37.321809703s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913109842 (10h43m37.344189973s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913131072 (10h43m37.344211193s) [seg_pos={0 0}, upload=11204]: Segment reached optimal threshold of 14 pieces. Aug 9 04:09:42.913224241 (10h43m37.344304372s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sF7TCu1pL8mrtjoPULJ2VnZQHcJSMmvou2w7YgF58ZHQcnfJT): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913270629 (10h43m37.34435075s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16aJgn61bhumVFmXfTv8YUi5JfZedtBqadv8yVcSEjgfgUxuDg): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913300577 (10h43m37.344380708s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xZEzok8AN2SUW6wups5Qug2TndRgrwkK2XMFvXfRyZcXFZRbU): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913331346 (10h43m37.344411477s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913347877 (10h43m37.344428008s) [seg_pos={0 0}, upload=11204]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 04:09:42.913400328 (10h43m37.344480459s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 04:09:42.913404055 (10h43m37.344484176s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:09:43.578329530 (10h43m38.009409661s) [upload=11204]: Done uploading segments... Aug 9 04:09:43.578350230 (10h43m38.009430361s) [upload=11204]: Upload finished. err: <nil> Duration: 5.831658074s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...X....X.........................X......X.....XXX........X.] duration: 16.298360964s FAIL (2025-08-09 04:15:59 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 16.298383748s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [..................X.............................X...........] duration: 796.248313ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-09 04:15:58 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 796.2668ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [......................................X........X............] duration: 1.849025883s FAIL (2025-08-09 04:15:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 04:07:10.334620288 (10h41m4.765700419s) [upload=11163]: Starting upload Aug 9 04:07:10.334666016 (10h41m4.765746147s) [upload=11163]: Done starting upload Aug 9 04:07:10.334726181 (10h41m4.765806312s) [upload=11163]: Uploading segments... Aug 9 04:07:10.334759073 (10h41m4.765839204s) [upload=11163]: Got next segment. Inline: true Aug 9 04:07:10.334763161 (10h41m4.765843292s) [upload=11163]: Waiting for error group managing segments... Aug 9 04:07:10.334796215 (10h41m4.765876336s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:07:10.334800333 (10h41m4.765880464s) [upload=11163]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 04:07:10.334802918 (10h41m4.765883049s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:07:11.359905316 (10h41m5.790985467s) [upload=11163]: Done uploading segments... Aug 9 04:07:11.359924452 (10h41m5.791004594s) [upload=11163]: Upload finished. err: <nil> Duration:* 1.849059868s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..................X................................X........] duration: 2.389977421s FAIL (2025-08-09 04:15:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.390028158s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [...................X.............................X..........] duration: 784.985147ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-09 04:15:00 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 785.009192ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X............X.............................X.........] duration: 5.484950548s FAIL (2025-08-09 04:14:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 523f30c4aaf94ddc9fbf8b4aeb9a36d20987df7a771e62b662b689d1413f1fbc != before 825f4c11ac74d2704e1d6c6d1b139ad0fcebca64bd571c7b27bbf33fb59231d4 ======LOGS====== Aug 9 04:09:40.375954558 (10h43m34.807034749s) [upload=11204]: Starting upload Aug 9 04:09:40.376058546 (10h43m34.807138677s) [upload=11204]: Done starting upload Aug 9 04:09:40.376218022 (10h43m34.807298153s) [upload=11204]: Uploading segments... Aug 9 04:09:40.376312062 (10h43m34.807392193s) [upload=11204]: Got next segment. Inline: false Aug 9 04:09:40.376344544 (10h43m34.807424675s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:09:40.376349013 (10h43m34.807429144s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 04:09:40.873631448 (10h43m35.304711579s) [seg_pos={0 0}, upload=11204]: Begin upload segment... Aug 9 04:09:40.873691684 (10h43m35.304771815s) [seg_pos={0 0}, upload=11204]: Done begin upload segment. Aug 9 04:09:40.873713004 (10h43m35.304793135s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.873755876 (10h43m35.304836007s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874170830 (10h43m35.305250951s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874868184 (10h43m35.305948315s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875229585 (10h43m35.306309706s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875522877 (10h43m35.306603008s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.877073413 (10h43m35.308153544s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.878880160 (10h43m35.309960281s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879375929 (10h43m35.31045605s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879497672 (10h43m35.310577803s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879783989 (10h43m35.310864121s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879811903 (10h43m35.310892034s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880014260 (10h43m35.311094391s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880236695 (10h43m35.311316826s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880492004 (10h43m35.311572135s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880711344 (10h43m35.311791465s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881026116 (10h43m35.312106247s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881320319 (10h43m35.31240045s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:42.031531447 (10h43m36.462611578s) [upload=11204]: Next returned nil segment Aug 9 04:09:42.031550193 (10h43m36.462630324s) [upload=11204]: Waiting for error group managing segments... Aug 9 04:09:42.321166913 (10h43m36.752247034s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.777697238 (10h43m37.208777369s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.789954881 (10h43m37.221035002s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.790550711 (10h43m37.221630842s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.812786204 (10h43m37.243866335s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.814899077 (10h43m37.245979208s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.823123292 (10h43m37.254203423s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.828425747 (10h43m37.259505878s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.834875747 (10h43m37.265955879s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.835243621 (10h43m37.266323742s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.874904975 (10h43m37.305985106s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.889634908 (10h43m37.320715039s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.890729572 (10h43m37.321809703s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913109842 (10h43m37.344189973s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913131072 (10h43m37.344211193s) [seg_pos={0 0}, upload=11204]: Segment reached optimal threshold of 14 pieces. Aug 9 04:09:42.913224241 (10h43m37.344304372s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sF7TCu1pL8mrtjoPULJ2VnZQHcJSMmvou2w7YgF58ZHQcnfJT): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913270629 (10h43m37.34435075s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16aJgn61bhumVFmXfTv8YUi5JfZedtBqadv8yVcSEjgfgUxuDg): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913300577 (10h43m37.344380708s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xZEzok8AN2SUW6wups5Qug2TndRgrwkK2XMFvXfRyZcXFZRbU): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913331346 (10h43m37.344411477s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913347877 (10h43m37.344428008s) [seg_pos={0 0}, upload=11204]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 04:09:42.913400328 (10h43m37.344480459s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 04:09:42.913404055 (10h43m37.344484176s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:09:43.578329530 (10h43m38.009409661s) [upload=11204]: Done uploading segments... Aug 9 04:09:43.578350230 (10h43m38.009430361s) [upload=11204]: Upload finished. err: <nil> Duration: 5.485041s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X........X...........] duration: 1.875684975s FAIL (2025-08-09 04:14:50 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 04:07:10.334620288 (10h41m4.765700419s) [upload=11163]: Starting upload Aug 9 04:07:10.334666016 (10h41m4.765746147s) [upload=11163]: Done starting upload Aug 9 04:07:10.334726181 (10h41m4.765806312s) [upload=11163]: Uploading segments... Aug 9 04:07:10.334759073 (10h41m4.765839204s) [upload=11163]: Got next segment. Inline: true Aug 9 04:07:10.334763161 (10h41m4.765843292s) [upload=11163]: Waiting for error group managing segments... Aug 9 04:07:10.334796215 (10h41m4.765876336s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:07:10.334800333 (10h41m4.765880464s) [upload=11163]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 04:07:10.334802918 (10h41m4.765883049s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:07:11.359905316 (10h41m5.790985467s) [upload=11163]: Done uploading segments... Aug 9 04:07:11.359924452 (10h41m5.791004594s) [upload=11163]: Upload finished. err: <nil> Duration:* 1.87572474s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [....X....X.........................X......X.....XXX........X] duration: 11.249923248s FAIL (2025-08-09 04:14:47 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 11.249950831s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [........X............X.............................X........] duration: 5.678916245s FAIL (2025-08-09 04:14:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 523f30c4aaf94ddc9fbf8b4aeb9a36d20987df7a771e62b662b689d1413f1fbc != before 825f4c11ac74d2704e1d6c6d1b139ad0fcebca64bd571c7b27bbf33fb59231d4 ======LOGS====== Aug 9 04:09:40.375954558 (10h43m34.807034749s) [upload=11204]: Starting upload Aug 9 04:09:40.376058546 (10h43m34.807138677s) [upload=11204]: Done starting upload Aug 9 04:09:40.376218022 (10h43m34.807298153s) [upload=11204]: Uploading segments... Aug 9 04:09:40.376312062 (10h43m34.807392193s) [upload=11204]: Got next segment. Inline: false Aug 9 04:09:40.376344544 (10h43m34.807424675s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:09:40.376349013 (10h43m34.807429144s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 04:09:40.873631448 (10h43m35.304711579s) [seg_pos={0 0}, upload=11204]: Begin upload segment... Aug 9 04:09:40.873691684 (10h43m35.304771815s) [seg_pos={0 0}, upload=11204]: Done begin upload segment. Aug 9 04:09:40.873713004 (10h43m35.304793135s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.873755876 (10h43m35.304836007s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874170830 (10h43m35.305250951s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874868184 (10h43m35.305948315s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875229585 (10h43m35.306309706s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875522877 (10h43m35.306603008s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.877073413 (10h43m35.308153544s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.878880160 (10h43m35.309960281s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879375929 (10h43m35.31045605s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879497672 (10h43m35.310577803s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879783989 (10h43m35.310864121s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879811903 (10h43m35.310892034s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880014260 (10h43m35.311094391s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880236695 (10h43m35.311316826s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880492004 (10h43m35.311572135s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880711344 (10h43m35.311791465s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881026116 (10h43m35.312106247s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881320319 (10h43m35.31240045s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:42.031531447 (10h43m36.462611578s) [upload=11204]: Next returned nil segment Aug 9 04:09:42.031550193 (10h43m36.462630324s) [upload=11204]: Waiting for error group managing segments... Aug 9 04:09:42.321166913 (10h43m36.752247034s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.777697238 (10h43m37.208777369s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.789954881 (10h43m37.221035002s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.790550711 (10h43m37.221630842s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.812786204 (10h43m37.243866335s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.814899077 (10h43m37.245979208s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.823123292 (10h43m37.254203423s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.828425747 (10h43m37.259505878s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.834875747 (10h43m37.265955879s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.835243621 (10h43m37.266323742s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.874904975 (10h43m37.305985106s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.889634908 (10h43m37.320715039s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.890729572 (10h43m37.321809703s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913109842 (10h43m37.344189973s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913131072 (10h43m37.344211193s) [seg_pos={0 0}, upload=11204]: Segment reached optimal threshold of 14 pieces. Aug 9 04:09:42.913224241 (10h43m37.344304372s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sF7TCu1pL8mrtjoPULJ2VnZQHcJSMmvou2w7YgF58ZHQcnfJT): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913270629 (10h43m37.34435075s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16aJgn61bhumVFmXfTv8YUi5JfZedtBqadv8yVcSEjgfgUxuDg): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913300577 (10h43m37.344380708s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xZEzok8AN2SUW6wups5Qug2TndRgrwkK2XMFvXfRyZcXFZRbU): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913331346 (10h43m37.344411477s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913347877 (10h43m37.344428008s) [seg_pos={0 0}, upload=11204]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 04:09:42.913400328 (10h43m37.344480459s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 04:09:42.913404055 (10h43m37.344484176s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:09:43.578329530 (10h43m38.009409661s) [upload=11204]: Done uploading segments... Aug 9 04:09:43.578350230 (10h43m38.009430361s) [upload=11204]: Upload finished. err: <nil> Duration: 5.678943577s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [...................X................................X.......] duration: 2.552014344s FAIL (2025-08-09 04:14:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.552030805s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [X...................X................................X......] duration: 3.200179994s FAIL (2025-08-09 04:14:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.200203049s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........X............X.............................X.......] duration: 5.917088097s FAIL (2025-08-09 04:14:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 523f30c4aaf94ddc9fbf8b4aeb9a36d20987df7a771e62b662b689d1413f1fbc != before 825f4c11ac74d2704e1d6c6d1b139ad0fcebca64bd571c7b27bbf33fb59231d4 ======LOGS====== Aug 9 04:09:40.375954558 (10h43m34.807034749s) [upload=11204]: Starting upload Aug 9 04:09:40.376058546 (10h43m34.807138677s) [upload=11204]: Done starting upload Aug 9 04:09:40.376218022 (10h43m34.807298153s) [upload=11204]: Uploading segments... Aug 9 04:09:40.376312062 (10h43m34.807392193s) [upload=11204]: Got next segment. Inline: false Aug 9 04:09:40.376344544 (10h43m34.807424675s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:09:40.376349013 (10h43m34.807429144s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 04:09:40.873631448 (10h43m35.304711579s) [seg_pos={0 0}, upload=11204]: Begin upload segment... Aug 9 04:09:40.873691684 (10h43m35.304771815s) [seg_pos={0 0}, upload=11204]: Done begin upload segment. Aug 9 04:09:40.873713004 (10h43m35.304793135s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.873755876 (10h43m35.304836007s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874170830 (10h43m35.305250951s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874868184 (10h43m35.305948315s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875229585 (10h43m35.306309706s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875522877 (10h43m35.306603008s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.877073413 (10h43m35.308153544s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.878880160 (10h43m35.309960281s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879375929 (10h43m35.31045605s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879497672 (10h43m35.310577803s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879783989 (10h43m35.310864121s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879811903 (10h43m35.310892034s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880014260 (10h43m35.311094391s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880236695 (10h43m35.311316826s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880492004 (10h43m35.311572135s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880711344 (10h43m35.311791465s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881026116 (10h43m35.312106247s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881320319 (10h43m35.31240045s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:42.031531447 (10h43m36.462611578s) [upload=11204]: Next returned nil segment Aug 9 04:09:42.031550193 (10h43m36.462630324s) [upload=11204]: Waiting for error group managing segments... Aug 9 04:09:42.321166913 (10h43m36.752247034s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.777697238 (10h43m37.208777369s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.789954881 (10h43m37.221035002s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.790550711 (10h43m37.221630842s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.812786204 (10h43m37.243866335s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.814899077 (10h43m37.245979208s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.823123292 (10h43m37.254203423s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.828425747 (10h43m37.259505878s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.834875747 (10h43m37.265955879s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.835243621 (10h43m37.266323742s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.874904975 (10h43m37.305985106s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.889634908 (10h43m37.320715039s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.890729572 (10h43m37.321809703s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913109842 (10h43m37.344189973s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913131072 (10h43m37.344211193s) [seg_pos={0 0}, upload=11204]: Segment reached optimal threshold of 14 pieces. Aug 9 04:09:42.913224241 (10h43m37.344304372s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sF7TCu1pL8mrtjoPULJ2VnZQHcJSMmvou2w7YgF58ZHQcnfJT): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913270629 (10h43m37.34435075s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16aJgn61bhumVFmXfTv8YUi5JfZedtBqadv8yVcSEjgfgUxuDg): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913300577 (10h43m37.344380708s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xZEzok8AN2SUW6wups5Qug2TndRgrwkK2XMFvXfRyZcXFZRbU): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913331346 (10h43m37.344411477s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913347877 (10h43m37.344428008s) [seg_pos={0 0}, upload=11204]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 04:09:42.913400328 (10h43m37.344480459s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 04:09:42.913404055 (10h43m37.344484176s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:09:43.578329530 (10h43m38.009409661s) [upload=11204]: Done uploading segments... Aug 9 04:09:43.578350230 (10h43m38.009430361s) [upload=11204]: Upload finished. err: <nil> Duration: 5.917109767s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [....................X.............................X.........] duration: 737.101698ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-09 04:13:58 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 737.125393ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................................X........X..........] duration: 1.914286015s FAIL (2025-08-09 04:13:44 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 04:07:10.334620288 (10h41m4.765700419s) [upload=11163]: Starting upload Aug 9 04:07:10.334666016 (10h41m4.765746147s) [upload=11163]: Done starting upload Aug 9 04:07:10.334726181 (10h41m4.765806312s) [upload=11163]: Uploading segments... Aug 9 04:07:10.334759073 (10h41m4.765839204s) [upload=11163]: Got next segment. Inline: true Aug 9 04:07:10.334763161 (10h41m4.765843292s) [upload=11163]: Waiting for error group managing segments... Aug 9 04:07:10.334796215 (10h41m4.765876336s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:07:10.334800333 (10h41m4.765880464s) [upload=11163]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 04:07:10.334802918 (10h41m4.765883049s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:07:11.359905316 (10h41m5.790985467s) [upload=11163]: Done uploading segments... Aug 9 04:07:11.359924452 (10h41m5.791004594s) [upload=11163]: Upload finished. err: <nil> Duration:* 1.914307845s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.....X....X.........................X......X.....XXX........] duration: 8.558275244s FAIL (2025-08-09 04:13:37 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 8.55828909s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 740.825102ms FAIL (2025-08-09 04:13:34 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 740.848579ms edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........X............X.............................X......] duration: 5.761517485s FAIL (2025-08-09 04:13:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 523f30c4aaf94ddc9fbf8b4aeb9a36d20987df7a771e62b662b689d1413f1fbc != before 825f4c11ac74d2704e1d6c6d1b139ad0fcebca64bd571c7b27bbf33fb59231d4 ======LOGS====== Aug 9 04:09:40.375954558 (10h43m34.807034749s) [upload=11204]: Starting upload Aug 9 04:09:40.376058546 (10h43m34.807138677s) [upload=11204]: Done starting upload Aug 9 04:09:40.376218022 (10h43m34.807298153s) [upload=11204]: Uploading segments... Aug 9 04:09:40.376312062 (10h43m34.807392193s) [upload=11204]: Got next segment. Inline: false Aug 9 04:09:40.376344544 (10h43m34.807424675s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:09:40.376349013 (10h43m34.807429144s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 04:09:40.873631448 (10h43m35.304711579s) [seg_pos={0 0}, upload=11204]: Begin upload segment... Aug 9 04:09:40.873691684 (10h43m35.304771815s) [seg_pos={0 0}, upload=11204]: Done begin upload segment. Aug 9 04:09:40.873713004 (10h43m35.304793135s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.873755876 (10h43m35.304836007s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874170830 (10h43m35.305250951s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874868184 (10h43m35.305948315s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875229585 (10h43m35.306309706s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875522877 (10h43m35.306603008s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.877073413 (10h43m35.308153544s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.878880160 (10h43m35.309960281s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879375929 (10h43m35.31045605s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879497672 (10h43m35.310577803s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879783989 (10h43m35.310864121s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879811903 (10h43m35.310892034s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880014260 (10h43m35.311094391s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880236695 (10h43m35.311316826s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880492004 (10h43m35.311572135s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880711344 (10h43m35.311791465s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881026116 (10h43m35.312106247s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881320319 (10h43m35.31240045s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:42.031531447 (10h43m36.462611578s) [upload=11204]: Next returned nil segment Aug 9 04:09:42.031550193 (10h43m36.462630324s) [upload=11204]: Waiting for error group managing segments... Aug 9 04:09:42.321166913 (10h43m36.752247034s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.777697238 (10h43m37.208777369s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.789954881 (10h43m37.221035002s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.790550711 (10h43m37.221630842s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.812786204 (10h43m37.243866335s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.814899077 (10h43m37.245979208s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.823123292 (10h43m37.254203423s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.828425747 (10h43m37.259505878s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.834875747 (10h43m37.265955879s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.835243621 (10h43m37.266323742s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.874904975 (10h43m37.305985106s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.889634908 (10h43m37.320715039s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.890729572 (10h43m37.321809703s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913109842 (10h43m37.344189973s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913131072 (10h43m37.344211193s) [seg_pos={0 0}, upload=11204]: Segment reached optimal threshold of 14 pieces. Aug 9 04:09:42.913224241 (10h43m37.344304372s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sF7TCu1pL8mrtjoPULJ2VnZQHcJSMmvou2w7YgF58ZHQcnfJT): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913270629 (10h43m37.34435075s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16aJgn61bhumVFmXfTv8YUi5JfZedtBqadv8yVcSEjgfgUxuDg): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913300577 (10h43m37.344380708s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xZEzok8AN2SUW6wups5Qug2TndRgrwkK2XMFvXfRyZcXFZRbU): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913331346 (10h43m37.344411477s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913347877 (10h43m37.344428008s) [seg_pos={0 0}, upload=11204]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 04:09:42.913400328 (10h43m37.344480459s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 04:09:42.913404055 (10h43m37.344484176s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:09:43.578329530 (10h43m38.009409661s) [upload=11204]: Done uploading segments... Aug 9 04:09:43.578350230 (10h43m38.009430361s) [upload=11204]: Upload finished. err: <nil> Duration: 5.761542422s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X........X.........] duration: 1.896026792s FAIL (2025-08-09 04:13:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 04:07:10.334620288 (10h41m4.765700419s) [upload=11163]: Starting upload Aug 9 04:07:10.334666016 (10h41m4.765746147s) [upload=11163]: Done starting upload Aug 9 04:07:10.334726181 (10h41m4.765806312s) [upload=11163]: Uploading segments... Aug 9 04:07:10.334759073 (10h41m4.765839204s) [upload=11163]: Got next segment. Inline: true Aug 9 04:07:10.334763161 (10h41m4.765843292s) [upload=11163]: Waiting for error group managing segments... Aug 9 04:07:10.334796215 (10h41m4.765876336s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:07:10.334800333 (10h41m4.765880464s) [upload=11163]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 04:07:10.334802918 (10h41m4.765883049s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:07:11.359905316 (10h41m5.790985467s) [upload=11163]: Done uploading segments... Aug 9 04:07:11.359924452 (10h41m5.791004594s) [upload=11163]: Upload finished. err: <nil> Duration:* 1.896057701s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........X............X.............................X.....] duration: 5.954465828s FAIL (2025-08-09 04:13:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 523f30c4aaf94ddc9fbf8b4aeb9a36d20987df7a771e62b662b689d1413f1fbc != before 825f4c11ac74d2704e1d6c6d1b139ad0fcebca64bd571c7b27bbf33fb59231d4 ======LOGS====== Aug 9 04:09:40.375954558 (10h43m34.807034749s) [upload=11204]: Starting upload Aug 9 04:09:40.376058546 (10h43m34.807138677s) [upload=11204]: Done starting upload Aug 9 04:09:40.376218022 (10h43m34.807298153s) [upload=11204]: Uploading segments... Aug 9 04:09:40.376312062 (10h43m34.807392193s) [upload=11204]: Got next segment. Inline: false Aug 9 04:09:40.376344544 (10h43m34.807424675s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:09:40.376349013 (10h43m34.807429144s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 04:09:40.873631448 (10h43m35.304711579s) [seg_pos={0 0}, upload=11204]: Begin upload segment... Aug 9 04:09:40.873691684 (10h43m35.304771815s) [seg_pos={0 0}, upload=11204]: Done begin upload segment. Aug 9 04:09:40.873713004 (10h43m35.304793135s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.873755876 (10h43m35.304836007s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874170830 (10h43m35.305250951s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874868184 (10h43m35.305948315s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875229585 (10h43m35.306309706s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875522877 (10h43m35.306603008s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.877073413 (10h43m35.308153544s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.878880160 (10h43m35.309960281s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879375929 (10h43m35.31045605s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879497672 (10h43m35.310577803s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879783989 (10h43m35.310864121s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879811903 (10h43m35.310892034s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880014260 (10h43m35.311094391s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880236695 (10h43m35.311316826s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880492004 (10h43m35.311572135s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880711344 (10h43m35.311791465s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881026116 (10h43m35.312106247s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881320319 (10h43m35.31240045s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:42.031531447 (10h43m36.462611578s) [upload=11204]: Next returned nil segment Aug 9 04:09:42.031550193 (10h43m36.462630324s) [upload=11204]: Waiting for error group managing segments... Aug 9 04:09:42.321166913 (10h43m36.752247034s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.777697238 (10h43m37.208777369s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.789954881 (10h43m37.221035002s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.790550711 (10h43m37.221630842s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.812786204 (10h43m37.243866335s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.814899077 (10h43m37.245979208s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.823123292 (10h43m37.254203423s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.828425747 (10h43m37.259505878s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.834875747 (10h43m37.265955879s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.835243621 (10h43m37.266323742s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.874904975 (10h43m37.305985106s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.889634908 (10h43m37.320715039s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.890729572 (10h43m37.321809703s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913109842 (10h43m37.344189973s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913131072 (10h43m37.344211193s) [seg_pos={0 0}, upload=11204]: Segment reached optimal threshold of 14 pieces. Aug 9 04:09:42.913224241 (10h43m37.344304372s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sF7TCu1pL8mrtjoPULJ2VnZQHcJSMmvou2w7YgF58ZHQcnfJT): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913270629 (10h43m37.34435075s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16aJgn61bhumVFmXfTv8YUi5JfZedtBqadv8yVcSEjgfgUxuDg): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913300577 (10h43m37.344380708s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xZEzok8AN2SUW6wups5Qug2TndRgrwkK2XMFvXfRyZcXFZRbU): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913331346 (10h43m37.344411477s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913347877 (10h43m37.344428008s) [seg_pos={0 0}, upload=11204]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 04:09:42.913400328 (10h43m37.344480459s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 04:09:42.913404055 (10h43m37.344484176s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:09:43.578329530 (10h43m38.009409661s) [upload=11204]: Done uploading segments... Aug 9 04:09:43.578350230 (10h43m38.009430361s) [upload=11204]: Upload finished. err: <nil> Duration: 5.954488683s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 731.847577ms FAIL (2025-08-09 04:13:10 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 731.871072ms edge authservice http us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [.X...................X................................X.....] duration: 2.453395396s FAIL (2025-08-09 04:13:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.45341774s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [......X....X.........................X......X.....XXX.......] duration: 6.161164364s FAIL (2025-08-09 04:12:57 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.161191146s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [.....................X.............................X........] duration: 845.476611ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-09 04:12:48 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 845.503554ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X........X........] duration: 1.89631977s FAIL (2025-08-09 04:12:38 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 04:07:10.334620288 (10h41m4.765700419s) [upload=11163]: Starting upload Aug 9 04:07:10.334666016 (10h41m4.765746147s) [upload=11163]: Done starting upload Aug 9 04:07:10.334726181 (10h41m4.765806312s) [upload=11163]: Uploading segments... Aug 9 04:07:10.334759073 (10h41m4.765839204s) [upload=11163]: Got next segment. Inline: true Aug 9 04:07:10.334763161 (10h41m4.765843292s) [upload=11163]: Waiting for error group managing segments... Aug 9 04:07:10.334796215 (10h41m4.765876336s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:07:10.334800333 (10h41m4.765880464s) [upload=11163]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 04:07:10.334802918 (10h41m4.765883049s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:07:11.359905316 (10h41m5.790985467s) [upload=11163]: Done uploading segments... Aug 9 04:07:11.359924452 (10h41m5.791004594s) [upload=11163]: Upload finished. err: <nil> Duration:* 1.896347281s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 753.341252ms FAIL (2025-08-09 04:12:31 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 753.368213ms edge authservice http us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.......X....X.........................X......X.....XXX......] duration: 6.369802461s FAIL (2025-08-09 04:12:30 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.369818551s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X........X.......] duration: 1.842455372s FAIL (2025-08-09 04:12:14 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 04:07:10.334620288 (10h41m4.765700419s) [upload=11163]: Starting upload Aug 9 04:07:10.334666016 (10h41m4.765746147s) [upload=11163]: Done starting upload Aug 9 04:07:10.334726181 (10h41m4.765806312s) [upload=11163]: Uploading segments... Aug 9 04:07:10.334759073 (10h41m4.765839204s) [upload=11163]: Got next segment. Inline: true Aug 9 04:07:10.334763161 (10h41m4.765843292s) [upload=11163]: Waiting for error group managing segments... Aug 9 04:07:10.334796215 (10h41m4.765876336s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:07:10.334800333 (10h41m4.765880464s) [upload=11163]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 04:07:10.334802918 (10h41m4.765883049s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:07:11.359905316 (10h41m5.790985467s) [upload=11163]: Done uploading segments... Aug 9 04:07:11.359924452 (10h41m5.791004594s) [upload=11163]: Upload finished. err: <nil> Duration:* 1.842497944s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [..X...................X................................X....] duration: 2.263261215s FAIL (2025-08-09 04:12:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.263283589s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [......................X.............................X.......] duration: 748.767339ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-09 04:12:12 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 748.793719ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............X............X.............................X....] duration: 5.621802011s FAIL (2025-08-09 04:12:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 523f30c4aaf94ddc9fbf8b4aeb9a36d20987df7a771e62b662b689d1413f1fbc != before 825f4c11ac74d2704e1d6c6d1b139ad0fcebca64bd571c7b27bbf33fb59231d4 ======LOGS====== Aug 9 04:09:40.375954558 (10h43m34.807034749s) [upload=11204]: Starting upload Aug 9 04:09:40.376058546 (10h43m34.807138677s) [upload=11204]: Done starting upload Aug 9 04:09:40.376218022 (10h43m34.807298153s) [upload=11204]: Uploading segments... Aug 9 04:09:40.376312062 (10h43m34.807392193s) [upload=11204]: Got next segment. Inline: false Aug 9 04:09:40.376344544 (10h43m34.807424675s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:09:40.376349013 (10h43m34.807429144s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 04:09:40.873631448 (10h43m35.304711579s) [seg_pos={0 0}, upload=11204]: Begin upload segment... Aug 9 04:09:40.873691684 (10h43m35.304771815s) [seg_pos={0 0}, upload=11204]: Done begin upload segment. Aug 9 04:09:40.873713004 (10h43m35.304793135s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.873755876 (10h43m35.304836007s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874170830 (10h43m35.305250951s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874868184 (10h43m35.305948315s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875229585 (10h43m35.306309706s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875522877 (10h43m35.306603008s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.877073413 (10h43m35.308153544s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.878880160 (10h43m35.309960281s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879375929 (10h43m35.31045605s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879497672 (10h43m35.310577803s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879783989 (10h43m35.310864121s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879811903 (10h43m35.310892034s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880014260 (10h43m35.311094391s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880236695 (10h43m35.311316826s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880492004 (10h43m35.311572135s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880711344 (10h43m35.311791465s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881026116 (10h43m35.312106247s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881320319 (10h43m35.31240045s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:42.031531447 (10h43m36.462611578s) [upload=11204]: Next returned nil segment Aug 9 04:09:42.031550193 (10h43m36.462630324s) [upload=11204]: Waiting for error group managing segments... Aug 9 04:09:42.321166913 (10h43m36.752247034s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.777697238 (10h43m37.208777369s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.789954881 (10h43m37.221035002s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.790550711 (10h43m37.221630842s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.812786204 (10h43m37.243866335s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.814899077 (10h43m37.245979208s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.823123292 (10h43m37.254203423s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.828425747 (10h43m37.259505878s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.834875747 (10h43m37.265955879s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.835243621 (10h43m37.266323742s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.874904975 (10h43m37.305985106s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.889634908 (10h43m37.320715039s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.890729572 (10h43m37.321809703s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913109842 (10h43m37.344189973s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913131072 (10h43m37.344211193s) [seg_pos={0 0}, upload=11204]: Segment reached optimal threshold of 14 pieces. Aug 9 04:09:42.913224241 (10h43m37.344304372s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sF7TCu1pL8mrtjoPULJ2VnZQHcJSMmvou2w7YgF58ZHQcnfJT): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913270629 (10h43m37.34435075s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16aJgn61bhumVFmXfTv8YUi5JfZedtBqadv8yVcSEjgfgUxuDg): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913300577 (10h43m37.344380708s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xZEzok8AN2SUW6wups5Qug2TndRgrwkK2XMFvXfRyZcXFZRbU): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913331346 (10h43m37.344411477s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913347877 (10h43m37.344428008s) [seg_pos={0 0}, upload=11204]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 04:09:42.913400328 (10h43m37.344480459s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 04:09:42.913404055 (10h43m37.344484176s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:09:43.578329530 (10h43m38.009409661s) [upload=11204]: Done uploading segments... Aug 9 04:09:43.578350230 (10h43m38.009430361s) [upload=11204]: Upload finished. err: <nil> Duration: 5.621829835s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 724.482263ms FAIL (2025-08-09 04:11:49 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 724.520968ms edge authservice http us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............X............X.............................X...] duration: 5.723675812s FAIL (2025-08-09 04:11:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 523f30c4aaf94ddc9fbf8b4aeb9a36d20987df7a771e62b662b689d1413f1fbc != before 825f4c11ac74d2704e1d6c6d1b139ad0fcebca64bd571c7b27bbf33fb59231d4 ======LOGS====== Aug 9 04:09:40.375954558 (10h43m34.807034749s) [upload=11204]: Starting upload Aug 9 04:09:40.376058546 (10h43m34.807138677s) [upload=11204]: Done starting upload Aug 9 04:09:40.376218022 (10h43m34.807298153s) [upload=11204]: Uploading segments... Aug 9 04:09:40.376312062 (10h43m34.807392193s) [upload=11204]: Got next segment. Inline: false Aug 9 04:09:40.376344544 (10h43m34.807424675s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:09:40.376349013 (10h43m34.807429144s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 9 04:09:40.873631448 (10h43m35.304711579s) [seg_pos={0 0}, upload=11204]: Begin upload segment... Aug 9 04:09:40.873691684 (10h43m35.304771815s) [seg_pos={0 0}, upload=11204]: Done begin upload segment. Aug 9 04:09:40.873713004 (10h43m35.304793135s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.873755876 (10h43m35.304836007s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874170830 (10h43m35.305250951s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.874868184 (10h43m35.305948315s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875229585 (10h43m35.306309706s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.875522877 (10h43m35.306603008s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.877073413 (10h43m35.308153544s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.878880160 (10h43m35.309960281s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879375929 (10h43m35.31045605s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879497672 (10h43m35.310577803s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879783989 (10h43m35.310864121s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.879811903 (10h43m35.310892034s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880014260 (10h43m35.311094391s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880236695 (10h43m35.311316826s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880492004 (10h43m35.311572135s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.880711344 (10h43m35.311791465s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881026116 (10h43m35.312106247s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:40.881320319 (10h43m35.31240045s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Uploading piece... Aug 9 04:09:42.031531447 (10h43m36.462611578s) [upload=11204]: Next returned nil segment Aug 9 04:09:42.031550193 (10h43m36.462630324s) [upload=11204]: Waiting for error group managing segments... Aug 9 04:09:42.321166913 (10h43m36.752247034s) [noise=true , address=64.49.13.155:4922 , piece_id=75BPBZM2XR75WQ47C7VML7C3HHH5F5OLIG43W7BWW64AIYYE5M5Q, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.777697238 (10h43m37.208777369s) [noise=true , address=104.36.82.37:4654 , piece_id=NXAQLGTTD4LRGDPIYYI6IOUC57VU53PP6KFG2XPN7AP5IZN5QCBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.789954881 (10h43m37.221035002s) [noise=true , address=104.36.82.45:4938 , piece_id=QPXKUT6TMKZEJN36TQZ4R6HBA7PUUJ7N7ZKVPWCJCK6IMOBR42DA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.790550711 (10h43m37.221630842s) [noise=true , address=104.36.82.23:4476 , piece_id=7CCFL3X6JSP6K4DRLTLYNZ3DUZKCSA7H3ZP6FE4CYBEDCOLAXVGQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.812786204 (10h43m37.243866335s) [noise=true , address=45.91.116.10:4091 , piece_id=ZJL6JPYPH6E57HQGPDVYK6DTJRCT2OSSDC4SMTR6MFFEBR46EJ5A, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.814899077 (10h43m37.245979208s) [noise=true , address=104.36.82.2:4764 , piece_id=TYPC5GMEUSZC4FEB2WYNJUQ2YO4L5YJWD6XLEUC76JIK66YFGERQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.823123292 (10h43m37.254203423s) [noise=true , address=104.36.82.12:4170 , piece_id=UYRCLVMBXLXKMTEKSD2EATIBTG6QOC74D6JJE57FSFQRHZC4A4WQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.828425747 (10h43m37.259505878s) [noise=true , address=104.36.82.8:4977 , piece_id=MMIUEEASH24EQZQL7O4SULJRJI2YEI3TMLDZ5JYPHTXH3YL3E6RA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.834875747 (10h43m37.265955879s) [noise=true , address=104.36.82.46:4485 , piece_id=KG43DCOTREIL6H7543JLVON7EAWLKDZ2543DS74F3FEB7V5HNTAA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.835243621 (10h43m37.266323742s) [noise=true , address=104.36.82.38:4698 , piece_id=23VBXJWLRR65OVOFN6CTNNVMVMGV6G7HLPWCEUPPH36NZWSTXQKQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.874904975 (10h43m37.305985106s) [noise=true , address=104.36.82.38:4699 , piece_id=I5OFQDBLUHSDKKXG2UHESPH4AIT7WORBXPQIWSLWUIZASETCJIBA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.889634908 (10h43m37.320715039s) [noise=true , address=104.36.82.47:4547 , piece_id=CCW6SQEVA2GQ6DASGC7ARHAT46WWV5K7ZXGWJJJF6R44A7JC3YWA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.890729572 (10h43m37.321809703s) [noise=true , address=104.36.82.58:4294 , piece_id=YSCRVWYYG7XPLKXU2TGRANB5FYX7IB4HXT7NEELCMBN2KF4D6NCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913109842 (10h43m37.344189973s) [noise=true , address=104.36.82.18:4400 , piece_id=AFZJ3U4AOVB36O2DPOPH7GORF4WBAKUSH6CKRI3OWWQK3PTPAWUA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: <nil> Aug 9 04:09:42.913131072 (10h43m37.344211193s) [seg_pos={0 0}, upload=11204]: Segment reached optimal threshold of 14 pieces. Aug 9 04:09:42.913224241 (10h43m37.344304372s) [noise=true , address=104.36.82.34:4557 , piece_id=EAEUBSIANHEQIQP2WZRXLAX7JRZK7VLZOVVCZWSQYYYCKQIYQWCQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sF7TCu1pL8mrtjoPULJ2VnZQHcJSMmvou2w7YgF58ZHQcnfJT): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913270629 (10h43m37.34435075s) [noise=true , address=104.36.82.60:4351 , piece_id=QCQAWTRTTIF6DUMUFID4T7U6IEHJPEHQZ4NREIRLJFIQRW54S4KQ, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16aJgn61bhumVFmXfTv8YUi5JfZedtBqadv8yVcSEjgfgUxuDg): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913300577 (10h43m37.344380708s) [noise=true , address=104.36.82.44:4897 , piece_id=3PS4W2SG7TJVFNA4PYWJK74CBVP2GP4ZLPPBL76VPVCQ3NEPWJXA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xZEzok8AN2SUW6wups5Qug2TndRgrwkK2XMFvXfRyZcXFZRbU): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913331346 (10h43m37.344411477s) [noise=true , address=104.36.82.62:4412 , piece_id=NNEJAEJUWXHS5RWKQPXRK753TXT5STSC5VAO3YPFGPBSBSPEB2FA, seg_pos={0 0}, upload=11204]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS): protocol: expected piece hash; context canceled; EOF Aug 9 04:09:42.913347877 (10h43m37.344428008s) [seg_pos={0 0}, upload=11204]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 9 04:09:42.913400328 (10h43m37.344480459s) [upload=11204]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 9 04:09:42.913404055 (10h43m37.344484176s) [upload=11204]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:09:43.578329530 (10h43m38.009409661s) [upload=11204]: Done uploading segments... Aug 9 04:09:43.578350230 (10h43m38.009430361s) [upload=11204]: Upload finished. err: <nil> Duration: 5.723698527s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [.......................X.............................X......] duration: 772.429872ms remote: "tcp:45.38.24.2:443" target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-09 04:11:29 UTC) orbiter warning: Get "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410": context deadline exceeded Duration: 772.453677ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [........X....X.........................X......X.....XXX.....] duration: 9.254698384s FAIL (2025-08-09 04:11:21 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 9.254720506s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 774.471359ms FAIL (2025-08-09 04:11:15 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": context deadline exceeded Duration: 774.505293ms edge authservice http us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 3 out of 60 is failed (threshold=3) [...X...................X................................X...] duration: 2.920912563s FAIL (2025-08-09 04:11:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 2.92093653s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................................X........X......] duration: 1.980156365s FAIL (2025-08-09 04:11:11 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 9 04:07:10.334620288 (10h41m4.765700419s) [upload=11163]: Starting upload Aug 9 04:07:10.334666016 (10h41m4.765746147s) [upload=11163]: Done starting upload Aug 9 04:07:10.334726181 (10h41m4.765806312s) [upload=11163]: Uploading segments... Aug 9 04:07:10.334759073 (10h41m4.765839204s) [upload=11163]: Got next segment. Inline: true Aug 9 04:07:10.334763161 (10h41m4.765843292s) [upload=11163]: Waiting for error group managing segments... Aug 9 04:07:10.334796215 (10h41m4.765876336s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 9 04:07:10.334800333 (10h41m4.765880464s) [upload=11163]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 9 04:07:10.334802918 (10h41m4.765883049s) [upload=11163]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 9 04:07:11.359905316 (10h41m5.790985467s) [upload=11163]: Done uploading segments... Aug 9 04:07:11.359924452 (10h41m5.791004594s) [upload=11163]: Upload finished. err: <nil> Duration:* 1.98018057s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.AuthServiceHTTP{"https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us_select_1)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)