Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [............X.....................X.......X.................] duration: 1.915677329s FAIL (2025-08-13 02:51:16 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.915706485s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X............X.......................................] duration: 7.825467711s FAIL (2025-08-13 02:51:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.82549363s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........................X.........................X..X....] duration: 6.082829741s FAIL (2025-08-13 02:51:09 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 02:48:06.110257765 (105h22m0.541337896s) [upload=109688]: Starting upload Aug 13 02:48:06.110305687 (105h22m0.541385808s) [upload=109688]: Done starting upload Aug 13 02:48:06.110375731 (105h22m0.541455912s) [upload=109688]: Uploading segments... Aug 13 02:48:06.110476684 (105h22m0.541556815s) [upload=109688]: Got next segment. Inline: false Aug 13 02:48:06.110516410 (105h22m0.541596541s) [upload=109688]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:48:06.110520628 (105h22m0.541600759s) [upload=109688]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:48:06.613080855 (105h22m1.044160986s) [seg_pos={0 0}, upload=109688]: Begin upload segment... Aug 13 02:48:06.613137654 (105h22m1.044217785s) [seg_pos={0 0}, upload=109688]: Done begin upload segment. Aug 13 02:48:06.613164856 (105h22m1.044244987s) [noise=true , address=79.127.205.246:4260 , piece_id=ZVSZBW6H4RCPM4EYXLUU4NCX7744BUXAPPEISIOH646JMQYJME5A, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.613216354 (105h22m1.044296485s) [noise=true , address=104.36.82.19:4408 , piece_id=66XCPGXBNRVOMIUMQ2JPTDUIMSELGB33GZLVFBZ2VBIFSPAUNNWA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.613764573 (105h22m1.044844704s) [noise=true , address=77.90.31.13:4648 , piece_id=ZNRSBXGTSEDHS7DRYXD5OB3I2KX57NLUO5DF2J6LV6WGTX72XQLQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.613951761 (105h22m1.045031892s) [noise=true , address=104.36.82.23:4457 , piece_id=N42RP7XTYFGU4XBF6DM43ZGDRG6Q2QY7R5CST3TOXE64PFZNFPNA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614168125 (105h22m1.045248246s) [noise=true , address=104.36.82.35:4608 , piece_id=73OYA3MRED57SAR4SYWMQZKOFWVQD2MFVR5DJHOT6LZKISIUREIA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614252657 (105h22m1.045332788s) [noise=true , address=104.36.82.16:4324 , piece_id=5N6RDFHSYVXKLTGFE7Z5KFHAUM6Q6PDAVQLTE5QTK43A5TKLZ5FA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614524798 (105h22m1.045604919s) [noise=true , address=104.36.82.3:4775 , piece_id=CKMJUCNO2EFPFM544235YYBNOVM52JOUZ2M2DUELFG26YJKDEHSQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614807949 (105h22m1.04588811s) [noise=true , address=104.36.82.61:4387 , piece_id=XEFYZWNTXOJJSSPVPBH7VR2F2KPWIM6O3SVESLP4CQGUVV5H4KRA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.615812291 (105h22m1.046892412s) [noise=true , address=104.36.82.38:4697 , piece_id=MOHXHEP5KTYIHBI4Y7MNJGSNB3HZNPC6G3TTD4MZHPL6OV7Z5HNQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.616160478 (105h22m1.047240609s) [noise=true , address=104.36.82.15:4267 , piece_id=E3QZTFWNXDJHJMFB4HGAX3M56577RJMEWRTGVUYH64GKS4LKMQHQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617090005 (105h22m1.048170136s) [noise=true , address=104.36.82.44:4884 , piece_id=5RBBWOYCAFYY3YRCR3ILNESVZG6YUBW2LK4JHG4OSXANOAKECMRA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617309065 (105h22m1.048389196s) [noise=true , address=77.90.31.19:4853 , piece_id=QQMEEOCVM5JIOMFBLLPX53IZLIH3YYXN5Q7HJ6RSMKJ4O3KR3JOQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617533804 (105h22m1.048613935s) [noise=true , address=104.36.82.10:4094 , piece_id=WFEUQSM3LXQYUCLPINGDW2LRGI64CDSDFGMJJSEVPY3JAYWJ6WAQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617748455 (105h22m1.048828586s) [noise=true , address=104.36.82.56:4233 , piece_id=7ZAFWYHLHJGPS762DVPVYCEXINDVS4RV4OUWLJLE2VTKNBNSAURQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617970119 (105h22m1.04905025s) [noise=true , address=77.90.31.15:4728 , piece_id=RE5BY5W72FHPG7MFPR74ODDDN7DH4OR7UE7DINSESKK3CSEQGM7A, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.618187555 (105h22m1.049267686s) [noise=true , address=64.49.13.170:4722 , piece_id=T6E4LILLRISQJTSE46U5QGLKZ4TR2S2UQSDDPGPYFRXRTQVP7BGA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.618393620 (105h22m1.049473751s) [noise=true , address=45.91.116.9:4266 , piece_id=LIHTPNRRCHXV4B67QBCOFTPCKO3XRXDPX5SM2WX2IZ2J6635H7JQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.618621346 (105h22m1.049701477s) [noise=true , address=104.36.82.36:4631 , piece_id=T5VOTZNRPERTBPJ5VN4PIHTSXMARBZSTMRPQ7BQI47TDAZ445UWA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:07.762699983 (105h22m2.193780114s) [upload=109688]: Next returned nil segment Aug 13 02:48:07.762717296 (105h22m2.193797427s) [upload=109688]: Waiting for error group managing segments... Aug 13 02:48:08.096700406 (105h22m2.527780537s) [noise=true , address=64.49.13.170:4722 , piece_id=T6E4LILLRISQJTSE46U5QGLKZ4TR2S2UQSDDPGPYFRXRTQVP7BGA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.566951245 (105h22m2.998031376s) [noise=true , address=79.127.205.246:4260 , piece_id=ZVSZBW6H4RCPM4EYXLUU4NCX7744BUXAPPEISIOH646JMQYJME5A, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.589306017 (105h22m3.020386148s) [noise=true , address=104.36.82.3:4775 , piece_id=CKMJUCNO2EFPFM544235YYBNOVM52JOUZ2M2DUELFG26YJKDEHSQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.595417891 (105h22m3.026498012s) [noise=true , address=104.36.82.36:4631 , piece_id=T5VOTZNRPERTBPJ5VN4PIHTSXMARBZSTMRPQ7BQI47TDAZ445UWA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.597625095 (105h22m3.028705226s) [noise=true , address=77.90.31.13:4648 , piece_id=ZNRSBXGTSEDHS7DRYXD5OB3I2KX57NLUO5DF2J6LV6WGTX72XQLQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.597881144 (105h22m3.028961275s) [noise=true , address=45.91.116.9:4266 , piece_id=LIHTPNRRCHXV4B67QBCOFTPCKO3XRXDPX5SM2WX2IZ2J6635H7JQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.609580850 (105h22m3.040660981s) [noise=true , address=104.36.82.56:4233 , piece_id=7ZAFWYHLHJGPS762DVPVYCEXINDVS4RV4OUWLJLE2VTKNBNSAURQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.610204523 (105h22m3.041284644s) [noise=true , address=104.36.82.35:4608 , piece_id=73OYA3MRED57SAR4SYWMQZKOFWVQD2MFVR5DJHOT6LZKISIUREIA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.618494986 (105h22m3.049575117s) [noise=true , address=104.36.82.16:4324 , piece_id=5N6RDFHSYVXKLTGFE7Z5KFHAUM6Q6PDAVQLTE5QTK43A5TKLZ5FA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.619842915 (105h22m3.050923046s) [noise=true , address=77.90.31.19:4853 , piece_id=QQMEEOCVM5JIOMFBLLPX53IZLIH3YYXN5Q7HJ6RSMKJ4O3KR3JOQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.623020644 (105h22m3.054100775s) [noise=true , address=104.36.82.23:4457 , piece_id=N42RP7XTYFGU4XBF6DM43ZGDRG6Q2QY7R5CST3TOXE64PFZNFPNA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.649340373 (105h22m3.080420514s) [noise=true , address=104.36.82.10:4094 , piece_id=WFEUQSM3LXQYUCLPINGDW2LRGI64CDSDFGMJJSEVPY3JAYWJ6WAQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.654253243 (105h22m3.085333374s) [noise=true , address=104.36.82.19:4408 , piece_id=66XCPGXBNRVOMIUMQ2JPTDUIMSELGB33GZLVFBZ2VBIFSPAUNNWA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.666354868 (105h22m3.097434999s) [noise=true , address=104.36.82.61:4387 , piece_id=XEFYZWNTXOJJSSPVPBH7VR2F2KPWIM6O3SVESLP4CQGUVV5H4KRA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.666382851 (105h22m3.097462982s) [seg_pos={0 0}, upload=109688]: Segment reached optimal threshold of 14 pieces. Aug 13 02:48:08.666526656 (105h22m3.097606797s) [noise=true , address=104.36.82.38:4697 , piece_id=MOHXHEP5KTYIHBI4Y7MNJGSNB3HZNPC6G3TTD4MZHPL6OV7Z5HNQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLiUHc5wogTx5spRu3aNfAh35g31kwXbEfn5aCJoHXUVDkhGU): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666629672 (105h22m3.097709805s) [noise=true , address=77.90.31.15:4728 , piece_id=RE5BY5W72FHPG7MFPR74ODDDN7DH4OR7UE7DINSESKK3CSEQGM7A, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666662105 (105h22m3.097742226s) [noise=true , address=104.36.82.44:4884 , piece_id=5RBBWOYCAFYY3YRCR3ILNESVZG6YUBW2LK4JHG4OSXANOAKECMRA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666719154 (105h22m3.097799295s) [noise=true , address=104.36.82.15:4267 , piece_id=E3QZTFWNXDJHJMFB4HGAX3M56577RJMEWRTGVUYH64GKS4LKMQHQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666745024 (105h22m3.097825165s) [seg_pos={0 0}, upload=109688]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:48:08.666829095 (105h22m3.097909246s) [upload=109688]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:48:08.666834735 (105h22m3.097914886s) [upload=109688]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:48:08.979301278 (105h22m3.410381399s) [upload=109688]: Done uploading segments... Aug 13 02:48:08.979320665 (105h22m3.410400796s) [upload=109688]: Upload finished. err: <nil> Duration: 6.082859188s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.............X.....................X.......X................] duration: 1.876978459s FAIL (2025-08-13 02:50:37 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.876996112s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........................X.........................X..X...] duration: 5.698752761s FAIL (2025-08-13 02:50:22 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 02:48:06.110257765 (105h22m0.541337896s) [upload=109688]: Starting upload Aug 13 02:48:06.110305687 (105h22m0.541385808s) [upload=109688]: Done starting upload Aug 13 02:48:06.110375731 (105h22m0.541455912s) [upload=109688]: Uploading segments... Aug 13 02:48:06.110476684 (105h22m0.541556815s) [upload=109688]: Got next segment. Inline: false Aug 13 02:48:06.110516410 (105h22m0.541596541s) [upload=109688]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:48:06.110520628 (105h22m0.541600759s) [upload=109688]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:48:06.613080855 (105h22m1.044160986s) [seg_pos={0 0}, upload=109688]: Begin upload segment... Aug 13 02:48:06.613137654 (105h22m1.044217785s) [seg_pos={0 0}, upload=109688]: Done begin upload segment. Aug 13 02:48:06.613164856 (105h22m1.044244987s) [noise=true , address=79.127.205.246:4260 , piece_id=ZVSZBW6H4RCPM4EYXLUU4NCX7744BUXAPPEISIOH646JMQYJME5A, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.613216354 (105h22m1.044296485s) [noise=true , address=104.36.82.19:4408 , piece_id=66XCPGXBNRVOMIUMQ2JPTDUIMSELGB33GZLVFBZ2VBIFSPAUNNWA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.613764573 (105h22m1.044844704s) [noise=true , address=77.90.31.13:4648 , piece_id=ZNRSBXGTSEDHS7DRYXD5OB3I2KX57NLUO5DF2J6LV6WGTX72XQLQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.613951761 (105h22m1.045031892s) [noise=true , address=104.36.82.23:4457 , piece_id=N42RP7XTYFGU4XBF6DM43ZGDRG6Q2QY7R5CST3TOXE64PFZNFPNA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614168125 (105h22m1.045248246s) [noise=true , address=104.36.82.35:4608 , piece_id=73OYA3MRED57SAR4SYWMQZKOFWVQD2MFVR5DJHOT6LZKISIUREIA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614252657 (105h22m1.045332788s) [noise=true , address=104.36.82.16:4324 , piece_id=5N6RDFHSYVXKLTGFE7Z5KFHAUM6Q6PDAVQLTE5QTK43A5TKLZ5FA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614524798 (105h22m1.045604919s) [noise=true , address=104.36.82.3:4775 , piece_id=CKMJUCNO2EFPFM544235YYBNOVM52JOUZ2M2DUELFG26YJKDEHSQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614807949 (105h22m1.04588811s) [noise=true , address=104.36.82.61:4387 , piece_id=XEFYZWNTXOJJSSPVPBH7VR2F2KPWIM6O3SVESLP4CQGUVV5H4KRA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.615812291 (105h22m1.046892412s) [noise=true , address=104.36.82.38:4697 , piece_id=MOHXHEP5KTYIHBI4Y7MNJGSNB3HZNPC6G3TTD4MZHPL6OV7Z5HNQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.616160478 (105h22m1.047240609s) [noise=true , address=104.36.82.15:4267 , piece_id=E3QZTFWNXDJHJMFB4HGAX3M56577RJMEWRTGVUYH64GKS4LKMQHQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617090005 (105h22m1.048170136s) [noise=true , address=104.36.82.44:4884 , piece_id=5RBBWOYCAFYY3YRCR3ILNESVZG6YUBW2LK4JHG4OSXANOAKECMRA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617309065 (105h22m1.048389196s) [noise=true , address=77.90.31.19:4853 , piece_id=QQMEEOCVM5JIOMFBLLPX53IZLIH3YYXN5Q7HJ6RSMKJ4O3KR3JOQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617533804 (105h22m1.048613935s) [noise=true , address=104.36.82.10:4094 , piece_id=WFEUQSM3LXQYUCLPINGDW2LRGI64CDSDFGMJJSEVPY3JAYWJ6WAQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617748455 (105h22m1.048828586s) [noise=true , address=104.36.82.56:4233 , piece_id=7ZAFWYHLHJGPS762DVPVYCEXINDVS4RV4OUWLJLE2VTKNBNSAURQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617970119 (105h22m1.04905025s) [noise=true , address=77.90.31.15:4728 , piece_id=RE5BY5W72FHPG7MFPR74ODDDN7DH4OR7UE7DINSESKK3CSEQGM7A, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.618187555 (105h22m1.049267686s) [noise=true , address=64.49.13.170:4722 , piece_id=T6E4LILLRISQJTSE46U5QGLKZ4TR2S2UQSDDPGPYFRXRTQVP7BGA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.618393620 (105h22m1.049473751s) [noise=true , address=45.91.116.9:4266 , piece_id=LIHTPNRRCHXV4B67QBCOFTPCKO3XRXDPX5SM2WX2IZ2J6635H7JQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.618621346 (105h22m1.049701477s) [noise=true , address=104.36.82.36:4631 , piece_id=T5VOTZNRPERTBPJ5VN4PIHTSXMARBZSTMRPQ7BQI47TDAZ445UWA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:07.762699983 (105h22m2.193780114s) [upload=109688]: Next returned nil segment Aug 13 02:48:07.762717296 (105h22m2.193797427s) [upload=109688]: Waiting for error group managing segments... Aug 13 02:48:08.096700406 (105h22m2.527780537s) [noise=true , address=64.49.13.170:4722 , piece_id=T6E4LILLRISQJTSE46U5QGLKZ4TR2S2UQSDDPGPYFRXRTQVP7BGA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.566951245 (105h22m2.998031376s) [noise=true , address=79.127.205.246:4260 , piece_id=ZVSZBW6H4RCPM4EYXLUU4NCX7744BUXAPPEISIOH646JMQYJME5A, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.589306017 (105h22m3.020386148s) [noise=true , address=104.36.82.3:4775 , piece_id=CKMJUCNO2EFPFM544235YYBNOVM52JOUZ2M2DUELFG26YJKDEHSQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.595417891 (105h22m3.026498012s) [noise=true , address=104.36.82.36:4631 , piece_id=T5VOTZNRPERTBPJ5VN4PIHTSXMARBZSTMRPQ7BQI47TDAZ445UWA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.597625095 (105h22m3.028705226s) [noise=true , address=77.90.31.13:4648 , piece_id=ZNRSBXGTSEDHS7DRYXD5OB3I2KX57NLUO5DF2J6LV6WGTX72XQLQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.597881144 (105h22m3.028961275s) [noise=true , address=45.91.116.9:4266 , piece_id=LIHTPNRRCHXV4B67QBCOFTPCKO3XRXDPX5SM2WX2IZ2J6635H7JQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.609580850 (105h22m3.040660981s) [noise=true , address=104.36.82.56:4233 , piece_id=7ZAFWYHLHJGPS762DVPVYCEXINDVS4RV4OUWLJLE2VTKNBNSAURQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.610204523 (105h22m3.041284644s) [noise=true , address=104.36.82.35:4608 , piece_id=73OYA3MRED57SAR4SYWMQZKOFWVQD2MFVR5DJHOT6LZKISIUREIA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.618494986 (105h22m3.049575117s) [noise=true , address=104.36.82.16:4324 , piece_id=5N6RDFHSYVXKLTGFE7Z5KFHAUM6Q6PDAVQLTE5QTK43A5TKLZ5FA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.619842915 (105h22m3.050923046s) [noise=true , address=77.90.31.19:4853 , piece_id=QQMEEOCVM5JIOMFBLLPX53IZLIH3YYXN5Q7HJ6RSMKJ4O3KR3JOQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.623020644 (105h22m3.054100775s) [noise=true , address=104.36.82.23:4457 , piece_id=N42RP7XTYFGU4XBF6DM43ZGDRG6Q2QY7R5CST3TOXE64PFZNFPNA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.649340373 (105h22m3.080420514s) [noise=true , address=104.36.82.10:4094 , piece_id=WFEUQSM3LXQYUCLPINGDW2LRGI64CDSDFGMJJSEVPY3JAYWJ6WAQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.654253243 (105h22m3.085333374s) [noise=true , address=104.36.82.19:4408 , piece_id=66XCPGXBNRVOMIUMQ2JPTDUIMSELGB33GZLVFBZ2VBIFSPAUNNWA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.666354868 (105h22m3.097434999s) [noise=true , address=104.36.82.61:4387 , piece_id=XEFYZWNTXOJJSSPVPBH7VR2F2KPWIM6O3SVESLP4CQGUVV5H4KRA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.666382851 (105h22m3.097462982s) [seg_pos={0 0}, upload=109688]: Segment reached optimal threshold of 14 pieces. Aug 13 02:48:08.666526656 (105h22m3.097606797s) [noise=true , address=104.36.82.38:4697 , piece_id=MOHXHEP5KTYIHBI4Y7MNJGSNB3HZNPC6G3TTD4MZHPL6OV7Z5HNQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLiUHc5wogTx5spRu3aNfAh35g31kwXbEfn5aCJoHXUVDkhGU): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666629672 (105h22m3.097709805s) [noise=true , address=77.90.31.15:4728 , piece_id=RE5BY5W72FHPG7MFPR74ODDDN7DH4OR7UE7DINSESKK3CSEQGM7A, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666662105 (105h22m3.097742226s) [noise=true , address=104.36.82.44:4884 , piece_id=5RBBWOYCAFYY3YRCR3ILNESVZG6YUBW2LK4JHG4OSXANOAKECMRA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666719154 (105h22m3.097799295s) [noise=true , address=104.36.82.15:4267 , piece_id=E3QZTFWNXDJHJMFB4HGAX3M56577RJMEWRTGVUYH64GKS4LKMQHQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666745024 (105h22m3.097825165s) [seg_pos={0 0}, upload=109688]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:48:08.666829095 (105h22m3.097909246s) [upload=109688]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:48:08.666834735 (105h22m3.097914886s) [upload=109688]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:48:08.979301278 (105h22m3.410381399s) [upload=109688]: Done uploading segments... Aug 13 02:48:08.979320665 (105h22m3.410400796s) [upload=109688]: Upload finished. err: <nil> Duration: 5.698774094s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X............X......................................] duration: 7.94753809s FAIL (2025-08-13 02:50:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.947569801s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [..............X.....................X.......X...............] duration: 2.093140814s FAIL (2025-08-13 02:49:32 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 2.093165311s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X............X.....................................] duration: 6.578970113s FAIL (2025-08-13 02:49:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.57898485s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [............................X.........................X..X..] duration: 5.871518671s FAIL (2025-08-13 02:49:09 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 02:48:06.110257765 (105h22m0.541337896s) [upload=109688]: Starting upload Aug 13 02:48:06.110305687 (105h22m0.541385808s) [upload=109688]: Done starting upload Aug 13 02:48:06.110375731 (105h22m0.541455912s) [upload=109688]: Uploading segments... Aug 13 02:48:06.110476684 (105h22m0.541556815s) [upload=109688]: Got next segment. Inline: false Aug 13 02:48:06.110516410 (105h22m0.541596541s) [upload=109688]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:48:06.110520628 (105h22m0.541600759s) [upload=109688]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:48:06.613080855 (105h22m1.044160986s) [seg_pos={0 0}, upload=109688]: Begin upload segment... Aug 13 02:48:06.613137654 (105h22m1.044217785s) [seg_pos={0 0}, upload=109688]: Done begin upload segment. Aug 13 02:48:06.613164856 (105h22m1.044244987s) [noise=true , address=79.127.205.246:4260 , piece_id=ZVSZBW6H4RCPM4EYXLUU4NCX7744BUXAPPEISIOH646JMQYJME5A, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.613216354 (105h22m1.044296485s) [noise=true , address=104.36.82.19:4408 , piece_id=66XCPGXBNRVOMIUMQ2JPTDUIMSELGB33GZLVFBZ2VBIFSPAUNNWA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.613764573 (105h22m1.044844704s) [noise=true , address=77.90.31.13:4648 , piece_id=ZNRSBXGTSEDHS7DRYXD5OB3I2KX57NLUO5DF2J6LV6WGTX72XQLQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.613951761 (105h22m1.045031892s) [noise=true , address=104.36.82.23:4457 , piece_id=N42RP7XTYFGU4XBF6DM43ZGDRG6Q2QY7R5CST3TOXE64PFZNFPNA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614168125 (105h22m1.045248246s) [noise=true , address=104.36.82.35:4608 , piece_id=73OYA3MRED57SAR4SYWMQZKOFWVQD2MFVR5DJHOT6LZKISIUREIA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614252657 (105h22m1.045332788s) [noise=true , address=104.36.82.16:4324 , piece_id=5N6RDFHSYVXKLTGFE7Z5KFHAUM6Q6PDAVQLTE5QTK43A5TKLZ5FA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614524798 (105h22m1.045604919s) [noise=true , address=104.36.82.3:4775 , piece_id=CKMJUCNO2EFPFM544235YYBNOVM52JOUZ2M2DUELFG26YJKDEHSQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614807949 (105h22m1.04588811s) [noise=true , address=104.36.82.61:4387 , piece_id=XEFYZWNTXOJJSSPVPBH7VR2F2KPWIM6O3SVESLP4CQGUVV5H4KRA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.615812291 (105h22m1.046892412s) [noise=true , address=104.36.82.38:4697 , piece_id=MOHXHEP5KTYIHBI4Y7MNJGSNB3HZNPC6G3TTD4MZHPL6OV7Z5HNQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.616160478 (105h22m1.047240609s) [noise=true , address=104.36.82.15:4267 , piece_id=E3QZTFWNXDJHJMFB4HGAX3M56577RJMEWRTGVUYH64GKS4LKMQHQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617090005 (105h22m1.048170136s) [noise=true , address=104.36.82.44:4884 , piece_id=5RBBWOYCAFYY3YRCR3ILNESVZG6YUBW2LK4JHG4OSXANOAKECMRA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617309065 (105h22m1.048389196s) [noise=true , address=77.90.31.19:4853 , piece_id=QQMEEOCVM5JIOMFBLLPX53IZLIH3YYXN5Q7HJ6RSMKJ4O3KR3JOQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617533804 (105h22m1.048613935s) [noise=true , address=104.36.82.10:4094 , piece_id=WFEUQSM3LXQYUCLPINGDW2LRGI64CDSDFGMJJSEVPY3JAYWJ6WAQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617748455 (105h22m1.048828586s) [noise=true , address=104.36.82.56:4233 , piece_id=7ZAFWYHLHJGPS762DVPVYCEXINDVS4RV4OUWLJLE2VTKNBNSAURQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617970119 (105h22m1.04905025s) [noise=true , address=77.90.31.15:4728 , piece_id=RE5BY5W72FHPG7MFPR74ODDDN7DH4OR7UE7DINSESKK3CSEQGM7A, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.618187555 (105h22m1.049267686s) [noise=true , address=64.49.13.170:4722 , piece_id=T6E4LILLRISQJTSE46U5QGLKZ4TR2S2UQSDDPGPYFRXRTQVP7BGA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.618393620 (105h22m1.049473751s) [noise=true , address=45.91.116.9:4266 , piece_id=LIHTPNRRCHXV4B67QBCOFTPCKO3XRXDPX5SM2WX2IZ2J6635H7JQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.618621346 (105h22m1.049701477s) [noise=true , address=104.36.82.36:4631 , piece_id=T5VOTZNRPERTBPJ5VN4PIHTSXMARBZSTMRPQ7BQI47TDAZ445UWA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:07.762699983 (105h22m2.193780114s) [upload=109688]: Next returned nil segment Aug 13 02:48:07.762717296 (105h22m2.193797427s) [upload=109688]: Waiting for error group managing segments... Aug 13 02:48:08.096700406 (105h22m2.527780537s) [noise=true , address=64.49.13.170:4722 , piece_id=T6E4LILLRISQJTSE46U5QGLKZ4TR2S2UQSDDPGPYFRXRTQVP7BGA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.566951245 (105h22m2.998031376s) [noise=true , address=79.127.205.246:4260 , piece_id=ZVSZBW6H4RCPM4EYXLUU4NCX7744BUXAPPEISIOH646JMQYJME5A, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.589306017 (105h22m3.020386148s) [noise=true , address=104.36.82.3:4775 , piece_id=CKMJUCNO2EFPFM544235YYBNOVM52JOUZ2M2DUELFG26YJKDEHSQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.595417891 (105h22m3.026498012s) [noise=true , address=104.36.82.36:4631 , piece_id=T5VOTZNRPERTBPJ5VN4PIHTSXMARBZSTMRPQ7BQI47TDAZ445UWA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.597625095 (105h22m3.028705226s) [noise=true , address=77.90.31.13:4648 , piece_id=ZNRSBXGTSEDHS7DRYXD5OB3I2KX57NLUO5DF2J6LV6WGTX72XQLQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.597881144 (105h22m3.028961275s) [noise=true , address=45.91.116.9:4266 , piece_id=LIHTPNRRCHXV4B67QBCOFTPCKO3XRXDPX5SM2WX2IZ2J6635H7JQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.609580850 (105h22m3.040660981s) [noise=true , address=104.36.82.56:4233 , piece_id=7ZAFWYHLHJGPS762DVPVYCEXINDVS4RV4OUWLJLE2VTKNBNSAURQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.610204523 (105h22m3.041284644s) [noise=true , address=104.36.82.35:4608 , piece_id=73OYA3MRED57SAR4SYWMQZKOFWVQD2MFVR5DJHOT6LZKISIUREIA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.618494986 (105h22m3.049575117s) [noise=true , address=104.36.82.16:4324 , piece_id=5N6RDFHSYVXKLTGFE7Z5KFHAUM6Q6PDAVQLTE5QTK43A5TKLZ5FA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.619842915 (105h22m3.050923046s) [noise=true , address=77.90.31.19:4853 , piece_id=QQMEEOCVM5JIOMFBLLPX53IZLIH3YYXN5Q7HJ6RSMKJ4O3KR3JOQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.623020644 (105h22m3.054100775s) [noise=true , address=104.36.82.23:4457 , piece_id=N42RP7XTYFGU4XBF6DM43ZGDRG6Q2QY7R5CST3TOXE64PFZNFPNA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.649340373 (105h22m3.080420514s) [noise=true , address=104.36.82.10:4094 , piece_id=WFEUQSM3LXQYUCLPINGDW2LRGI64CDSDFGMJJSEVPY3JAYWJ6WAQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.654253243 (105h22m3.085333374s) [noise=true , address=104.36.82.19:4408 , piece_id=66XCPGXBNRVOMIUMQ2JPTDUIMSELGB33GZLVFBZ2VBIFSPAUNNWA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.666354868 (105h22m3.097434999s) [noise=true , address=104.36.82.61:4387 , piece_id=XEFYZWNTXOJJSSPVPBH7VR2F2KPWIM6O3SVESLP4CQGUVV5H4KRA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.666382851 (105h22m3.097462982s) [seg_pos={0 0}, upload=109688]: Segment reached optimal threshold of 14 pieces. Aug 13 02:48:08.666526656 (105h22m3.097606797s) [noise=true , address=104.36.82.38:4697 , piece_id=MOHXHEP5KTYIHBI4Y7MNJGSNB3HZNPC6G3TTD4MZHPL6OV7Z5HNQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLiUHc5wogTx5spRu3aNfAh35g31kwXbEfn5aCJoHXUVDkhGU): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666629672 (105h22m3.097709805s) [noise=true , address=77.90.31.15:4728 , piece_id=RE5BY5W72FHPG7MFPR74ODDDN7DH4OR7UE7DINSESKK3CSEQGM7A, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666662105 (105h22m3.097742226s) [noise=true , address=104.36.82.44:4884 , piece_id=5RBBWOYCAFYY3YRCR3ILNESVZG6YUBW2LK4JHG4OSXANOAKECMRA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666719154 (105h22m3.097799295s) [noise=true , address=104.36.82.15:4267 , piece_id=E3QZTFWNXDJHJMFB4HGAX3M56577RJMEWRTGVUYH64GKS4LKMQHQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666745024 (105h22m3.097825165s) [seg_pos={0 0}, upload=109688]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:48:08.666829095 (105h22m3.097909246s) [upload=109688]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:48:08.666834735 (105h22m3.097914886s) [upload=109688]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:48:08.979301278 (105h22m3.410381399s) [upload=109688]: Done uploading segments... Aug 13 02:48:08.979320665 (105h22m3.410400796s) [upload=109688]: Upload finished. err: <nil> Duration: 5.871543238s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.............................X.........................X..X.] duration: 5.869928434s FAIL (2025-08-13 02:48:33 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 02:48:06.110257765 (105h22m0.541337896s) [upload=109688]: Starting upload Aug 13 02:48:06.110305687 (105h22m0.541385808s) [upload=109688]: Done starting upload Aug 13 02:48:06.110375731 (105h22m0.541455912s) [upload=109688]: Uploading segments... Aug 13 02:48:06.110476684 (105h22m0.541556815s) [upload=109688]: Got next segment. Inline: false Aug 13 02:48:06.110516410 (105h22m0.541596541s) [upload=109688]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:48:06.110520628 (105h22m0.541600759s) [upload=109688]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:48:06.613080855 (105h22m1.044160986s) [seg_pos={0 0}, upload=109688]: Begin upload segment... Aug 13 02:48:06.613137654 (105h22m1.044217785s) [seg_pos={0 0}, upload=109688]: Done begin upload segment. Aug 13 02:48:06.613164856 (105h22m1.044244987s) [noise=true , address=79.127.205.246:4260 , piece_id=ZVSZBW6H4RCPM4EYXLUU4NCX7744BUXAPPEISIOH646JMQYJME5A, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.613216354 (105h22m1.044296485s) [noise=true , address=104.36.82.19:4408 , piece_id=66XCPGXBNRVOMIUMQ2JPTDUIMSELGB33GZLVFBZ2VBIFSPAUNNWA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.613764573 (105h22m1.044844704s) [noise=true , address=77.90.31.13:4648 , piece_id=ZNRSBXGTSEDHS7DRYXD5OB3I2KX57NLUO5DF2J6LV6WGTX72XQLQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.613951761 (105h22m1.045031892s) [noise=true , address=104.36.82.23:4457 , piece_id=N42RP7XTYFGU4XBF6DM43ZGDRG6Q2QY7R5CST3TOXE64PFZNFPNA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614168125 (105h22m1.045248246s) [noise=true , address=104.36.82.35:4608 , piece_id=73OYA3MRED57SAR4SYWMQZKOFWVQD2MFVR5DJHOT6LZKISIUREIA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614252657 (105h22m1.045332788s) [noise=true , address=104.36.82.16:4324 , piece_id=5N6RDFHSYVXKLTGFE7Z5KFHAUM6Q6PDAVQLTE5QTK43A5TKLZ5FA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614524798 (105h22m1.045604919s) [noise=true , address=104.36.82.3:4775 , piece_id=CKMJUCNO2EFPFM544235YYBNOVM52JOUZ2M2DUELFG26YJKDEHSQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614807949 (105h22m1.04588811s) [noise=true , address=104.36.82.61:4387 , piece_id=XEFYZWNTXOJJSSPVPBH7VR2F2KPWIM6O3SVESLP4CQGUVV5H4KRA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.615812291 (105h22m1.046892412s) [noise=true , address=104.36.82.38:4697 , piece_id=MOHXHEP5KTYIHBI4Y7MNJGSNB3HZNPC6G3TTD4MZHPL6OV7Z5HNQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.616160478 (105h22m1.047240609s) [noise=true , address=104.36.82.15:4267 , piece_id=E3QZTFWNXDJHJMFB4HGAX3M56577RJMEWRTGVUYH64GKS4LKMQHQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617090005 (105h22m1.048170136s) [noise=true , address=104.36.82.44:4884 , piece_id=5RBBWOYCAFYY3YRCR3ILNESVZG6YUBW2LK4JHG4OSXANOAKECMRA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617309065 (105h22m1.048389196s) [noise=true , address=77.90.31.19:4853 , piece_id=QQMEEOCVM5JIOMFBLLPX53IZLIH3YYXN5Q7HJ6RSMKJ4O3KR3JOQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617533804 (105h22m1.048613935s) [noise=true , address=104.36.82.10:4094 , piece_id=WFEUQSM3LXQYUCLPINGDW2LRGI64CDSDFGMJJSEVPY3JAYWJ6WAQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617748455 (105h22m1.048828586s) [noise=true , address=104.36.82.56:4233 , piece_id=7ZAFWYHLHJGPS762DVPVYCEXINDVS4RV4OUWLJLE2VTKNBNSAURQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617970119 (105h22m1.04905025s) [noise=true , address=77.90.31.15:4728 , piece_id=RE5BY5W72FHPG7MFPR74ODDDN7DH4OR7UE7DINSESKK3CSEQGM7A, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.618187555 (105h22m1.049267686s) [noise=true , address=64.49.13.170:4722 , piece_id=T6E4LILLRISQJTSE46U5QGLKZ4TR2S2UQSDDPGPYFRXRTQVP7BGA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.618393620 (105h22m1.049473751s) [noise=true , address=45.91.116.9:4266 , piece_id=LIHTPNRRCHXV4B67QBCOFTPCKO3XRXDPX5SM2WX2IZ2J6635H7JQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.618621346 (105h22m1.049701477s) [noise=true , address=104.36.82.36:4631 , piece_id=T5VOTZNRPERTBPJ5VN4PIHTSXMARBZSTMRPQ7BQI47TDAZ445UWA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:07.762699983 (105h22m2.193780114s) [upload=109688]: Next returned nil segment Aug 13 02:48:07.762717296 (105h22m2.193797427s) [upload=109688]: Waiting for error group managing segments... Aug 13 02:48:08.096700406 (105h22m2.527780537s) [noise=true , address=64.49.13.170:4722 , piece_id=T6E4LILLRISQJTSE46U5QGLKZ4TR2S2UQSDDPGPYFRXRTQVP7BGA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.566951245 (105h22m2.998031376s) [noise=true , address=79.127.205.246:4260 , piece_id=ZVSZBW6H4RCPM4EYXLUU4NCX7744BUXAPPEISIOH646JMQYJME5A, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.589306017 (105h22m3.020386148s) [noise=true , address=104.36.82.3:4775 , piece_id=CKMJUCNO2EFPFM544235YYBNOVM52JOUZ2M2DUELFG26YJKDEHSQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.595417891 (105h22m3.026498012s) [noise=true , address=104.36.82.36:4631 , piece_id=T5VOTZNRPERTBPJ5VN4PIHTSXMARBZSTMRPQ7BQI47TDAZ445UWA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.597625095 (105h22m3.028705226s) [noise=true , address=77.90.31.13:4648 , piece_id=ZNRSBXGTSEDHS7DRYXD5OB3I2KX57NLUO5DF2J6LV6WGTX72XQLQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.597881144 (105h22m3.028961275s) [noise=true , address=45.91.116.9:4266 , piece_id=LIHTPNRRCHXV4B67QBCOFTPCKO3XRXDPX5SM2WX2IZ2J6635H7JQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.609580850 (105h22m3.040660981s) [noise=true , address=104.36.82.56:4233 , piece_id=7ZAFWYHLHJGPS762DVPVYCEXINDVS4RV4OUWLJLE2VTKNBNSAURQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.610204523 (105h22m3.041284644s) [noise=true , address=104.36.82.35:4608 , piece_id=73OYA3MRED57SAR4SYWMQZKOFWVQD2MFVR5DJHOT6LZKISIUREIA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.618494986 (105h22m3.049575117s) [noise=true , address=104.36.82.16:4324 , piece_id=5N6RDFHSYVXKLTGFE7Z5KFHAUM6Q6PDAVQLTE5QTK43A5TKLZ5FA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.619842915 (105h22m3.050923046s) [noise=true , address=77.90.31.19:4853 , piece_id=QQMEEOCVM5JIOMFBLLPX53IZLIH3YYXN5Q7HJ6RSMKJ4O3KR3JOQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.623020644 (105h22m3.054100775s) [noise=true , address=104.36.82.23:4457 , piece_id=N42RP7XTYFGU4XBF6DM43ZGDRG6Q2QY7R5CST3TOXE64PFZNFPNA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.649340373 (105h22m3.080420514s) [noise=true , address=104.36.82.10:4094 , piece_id=WFEUQSM3LXQYUCLPINGDW2LRGI64CDSDFGMJJSEVPY3JAYWJ6WAQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.654253243 (105h22m3.085333374s) [noise=true , address=104.36.82.19:4408 , piece_id=66XCPGXBNRVOMIUMQ2JPTDUIMSELGB33GZLVFBZ2VBIFSPAUNNWA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.666354868 (105h22m3.097434999s) [noise=true , address=104.36.82.61:4387 , piece_id=XEFYZWNTXOJJSSPVPBH7VR2F2KPWIM6O3SVESLP4CQGUVV5H4KRA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.666382851 (105h22m3.097462982s) [seg_pos={0 0}, upload=109688]: Segment reached optimal threshold of 14 pieces. Aug 13 02:48:08.666526656 (105h22m3.097606797s) [noise=true , address=104.36.82.38:4697 , piece_id=MOHXHEP5KTYIHBI4Y7MNJGSNB3HZNPC6G3TTD4MZHPL6OV7Z5HNQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLiUHc5wogTx5spRu3aNfAh35g31kwXbEfn5aCJoHXUVDkhGU): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666629672 (105h22m3.097709805s) [noise=true , address=77.90.31.15:4728 , piece_id=RE5BY5W72FHPG7MFPR74ODDDN7DH4OR7UE7DINSESKK3CSEQGM7A, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666662105 (105h22m3.097742226s) [noise=true , address=104.36.82.44:4884 , piece_id=5RBBWOYCAFYY3YRCR3ILNESVZG6YUBW2LK4JHG4OSXANOAKECMRA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666719154 (105h22m3.097799295s) [noise=true , address=104.36.82.15:4267 , piece_id=E3QZTFWNXDJHJMFB4HGAX3M56577RJMEWRTGVUYH64GKS4LKMQHQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666745024 (105h22m3.097825165s) [seg_pos={0 0}, upload=109688]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:48:08.666829095 (105h22m3.097909246s) [upload=109688]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:48:08.666834735 (105h22m3.097914886s) [upload=109688]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:48:08.979301278 (105h22m3.410381399s) [upload=109688]: Done uploading segments... Aug 13 02:48:08.979320665 (105h22m3.410400796s) [upload=109688]: Upload finished. err: <nil> Duration: 5.869951229s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [...............X.....................X.......X..............] duration: 1.860153309s FAIL (2025-08-13 02:48:33 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.860171042s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X............X....................................] duration: 7.326887324s FAIL (2025-08-13 02:48:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.326910296s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [..............................X.........................X..X] duration: 4.189941584s FAIL (2025-08-13 02:48:09 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 13 02:48:06.110257765 (105h22m0.541337896s) [upload=109688]: Starting upload Aug 13 02:48:06.110305687 (105h22m0.541385808s) [upload=109688]: Done starting upload Aug 13 02:48:06.110375731 (105h22m0.541455912s) [upload=109688]: Uploading segments... Aug 13 02:48:06.110476684 (105h22m0.541556815s) [upload=109688]: Got next segment. Inline: false Aug 13 02:48:06.110516410 (105h22m0.541596541s) [upload=109688]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:48:06.110520628 (105h22m0.541600759s) [upload=109688]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:48:06.613080855 (105h22m1.044160986s) [seg_pos={0 0}, upload=109688]: Begin upload segment... Aug 13 02:48:06.613137654 (105h22m1.044217785s) [seg_pos={0 0}, upload=109688]: Done begin upload segment. Aug 13 02:48:06.613164856 (105h22m1.044244987s) [noise=true , address=79.127.205.246:4260 , piece_id=ZVSZBW6H4RCPM4EYXLUU4NCX7744BUXAPPEISIOH646JMQYJME5A, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.613216354 (105h22m1.044296485s) [noise=true , address=104.36.82.19:4408 , piece_id=66XCPGXBNRVOMIUMQ2JPTDUIMSELGB33GZLVFBZ2VBIFSPAUNNWA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.613764573 (105h22m1.044844704s) [noise=true , address=77.90.31.13:4648 , piece_id=ZNRSBXGTSEDHS7DRYXD5OB3I2KX57NLUO5DF2J6LV6WGTX72XQLQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.613951761 (105h22m1.045031892s) [noise=true , address=104.36.82.23:4457 , piece_id=N42RP7XTYFGU4XBF6DM43ZGDRG6Q2QY7R5CST3TOXE64PFZNFPNA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614168125 (105h22m1.045248246s) [noise=true , address=104.36.82.35:4608 , piece_id=73OYA3MRED57SAR4SYWMQZKOFWVQD2MFVR5DJHOT6LZKISIUREIA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614252657 (105h22m1.045332788s) [noise=true , address=104.36.82.16:4324 , piece_id=5N6RDFHSYVXKLTGFE7Z5KFHAUM6Q6PDAVQLTE5QTK43A5TKLZ5FA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614524798 (105h22m1.045604919s) [noise=true , address=104.36.82.3:4775 , piece_id=CKMJUCNO2EFPFM544235YYBNOVM52JOUZ2M2DUELFG26YJKDEHSQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.614807949 (105h22m1.04588811s) [noise=true , address=104.36.82.61:4387 , piece_id=XEFYZWNTXOJJSSPVPBH7VR2F2KPWIM6O3SVESLP4CQGUVV5H4KRA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.615812291 (105h22m1.046892412s) [noise=true , address=104.36.82.38:4697 , piece_id=MOHXHEP5KTYIHBI4Y7MNJGSNB3HZNPC6G3TTD4MZHPL6OV7Z5HNQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.616160478 (105h22m1.047240609s) [noise=true , address=104.36.82.15:4267 , piece_id=E3QZTFWNXDJHJMFB4HGAX3M56577RJMEWRTGVUYH64GKS4LKMQHQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617090005 (105h22m1.048170136s) [noise=true , address=104.36.82.44:4884 , piece_id=5RBBWOYCAFYY3YRCR3ILNESVZG6YUBW2LK4JHG4OSXANOAKECMRA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617309065 (105h22m1.048389196s) [noise=true , address=77.90.31.19:4853 , piece_id=QQMEEOCVM5JIOMFBLLPX53IZLIH3YYXN5Q7HJ6RSMKJ4O3KR3JOQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617533804 (105h22m1.048613935s) [noise=true , address=104.36.82.10:4094 , piece_id=WFEUQSM3LXQYUCLPINGDW2LRGI64CDSDFGMJJSEVPY3JAYWJ6WAQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617748455 (105h22m1.048828586s) [noise=true , address=104.36.82.56:4233 , piece_id=7ZAFWYHLHJGPS762DVPVYCEXINDVS4RV4OUWLJLE2VTKNBNSAURQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.617970119 (105h22m1.04905025s) [noise=true , address=77.90.31.15:4728 , piece_id=RE5BY5W72FHPG7MFPR74ODDDN7DH4OR7UE7DINSESKK3CSEQGM7A, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.618187555 (105h22m1.049267686s) [noise=true , address=64.49.13.170:4722 , piece_id=T6E4LILLRISQJTSE46U5QGLKZ4TR2S2UQSDDPGPYFRXRTQVP7BGA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.618393620 (105h22m1.049473751s) [noise=true , address=45.91.116.9:4266 , piece_id=LIHTPNRRCHXV4B67QBCOFTPCKO3XRXDPX5SM2WX2IZ2J6635H7JQ, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:06.618621346 (105h22m1.049701477s) [noise=true , address=104.36.82.36:4631 , piece_id=T5VOTZNRPERTBPJ5VN4PIHTSXMARBZSTMRPQ7BQI47TDAZ445UWA, seg_pos={0 0}, upload=109688]: Uploading piece... Aug 13 02:48:07.762699983 (105h22m2.193780114s) [upload=109688]: Next returned nil segment Aug 13 02:48:07.762717296 (105h22m2.193797427s) [upload=109688]: Waiting for error group managing segments... Aug 13 02:48:08.096700406 (105h22m2.527780537s) [noise=true , address=64.49.13.170:4722 , piece_id=T6E4LILLRISQJTSE46U5QGLKZ4TR2S2UQSDDPGPYFRXRTQVP7BGA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.566951245 (105h22m2.998031376s) [noise=true , address=79.127.205.246:4260 , piece_id=ZVSZBW6H4RCPM4EYXLUU4NCX7744BUXAPPEISIOH646JMQYJME5A, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.589306017 (105h22m3.020386148s) [noise=true , address=104.36.82.3:4775 , piece_id=CKMJUCNO2EFPFM544235YYBNOVM52JOUZ2M2DUELFG26YJKDEHSQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.595417891 (105h22m3.026498012s) [noise=true , address=104.36.82.36:4631 , piece_id=T5VOTZNRPERTBPJ5VN4PIHTSXMARBZSTMRPQ7BQI47TDAZ445UWA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.597625095 (105h22m3.028705226s) [noise=true , address=77.90.31.13:4648 , piece_id=ZNRSBXGTSEDHS7DRYXD5OB3I2KX57NLUO5DF2J6LV6WGTX72XQLQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.597881144 (105h22m3.028961275s) [noise=true , address=45.91.116.9:4266 , piece_id=LIHTPNRRCHXV4B67QBCOFTPCKO3XRXDPX5SM2WX2IZ2J6635H7JQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.609580850 (105h22m3.040660981s) [noise=true , address=104.36.82.56:4233 , piece_id=7ZAFWYHLHJGPS762DVPVYCEXINDVS4RV4OUWLJLE2VTKNBNSAURQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.610204523 (105h22m3.041284644s) [noise=true , address=104.36.82.35:4608 , piece_id=73OYA3MRED57SAR4SYWMQZKOFWVQD2MFVR5DJHOT6LZKISIUREIA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.618494986 (105h22m3.049575117s) [noise=true , address=104.36.82.16:4324 , piece_id=5N6RDFHSYVXKLTGFE7Z5KFHAUM6Q6PDAVQLTE5QTK43A5TKLZ5FA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.619842915 (105h22m3.050923046s) [noise=true , address=77.90.31.19:4853 , piece_id=QQMEEOCVM5JIOMFBLLPX53IZLIH3YYXN5Q7HJ6RSMKJ4O3KR3JOQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.623020644 (105h22m3.054100775s) [noise=true , address=104.36.82.23:4457 , piece_id=N42RP7XTYFGU4XBF6DM43ZGDRG6Q2QY7R5CST3TOXE64PFZNFPNA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.649340373 (105h22m3.080420514s) [noise=true , address=104.36.82.10:4094 , piece_id=WFEUQSM3LXQYUCLPINGDW2LRGI64CDSDFGMJJSEVPY3JAYWJ6WAQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.654253243 (105h22m3.085333374s) [noise=true , address=104.36.82.19:4408 , piece_id=66XCPGXBNRVOMIUMQ2JPTDUIMSELGB33GZLVFBZ2VBIFSPAUNNWA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.666354868 (105h22m3.097434999s) [noise=true , address=104.36.82.61:4387 , piece_id=XEFYZWNTXOJJSSPVPBH7VR2F2KPWIM6O3SVESLP4CQGUVV5H4KRA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: <nil> Aug 13 02:48:08.666382851 (105h22m3.097462982s) [seg_pos={0 0}, upload=109688]: Segment reached optimal threshold of 14 pieces. Aug 13 02:48:08.666526656 (105h22m3.097606797s) [noise=true , address=104.36.82.38:4697 , piece_id=MOHXHEP5KTYIHBI4Y7MNJGSNB3HZNPC6G3TTD4MZHPL6OV7Z5HNQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mLiUHc5wogTx5spRu3aNfAh35g31kwXbEfn5aCJoHXUVDkhGU): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666629672 (105h22m3.097709805s) [noise=true , address=77.90.31.15:4728 , piece_id=RE5BY5W72FHPG7MFPR74ODDDN7DH4OR7UE7DINSESKK3CSEQGM7A, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:152bVnKGB6XVkvkjrWDsivCMxEPXq3tesh1srTzXZBcuEohFDo): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666662105 (105h22m3.097742226s) [noise=true , address=104.36.82.44:4884 , piece_id=5RBBWOYCAFYY3YRCR3ILNESVZG6YUBW2LK4JHG4OSXANOAKECMRA, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1qGXdnm83P4eMnCtZdRt2x5yAWP7pQafWwoPG72daZJVR9pyKG): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666719154 (105h22m3.097799295s) [noise=true , address=104.36.82.15:4267 , piece_id=E3QZTFWNXDJHJMFB4HGAX3M56577RJMEWRTGVUYH64GKS4LKMQHQ, seg_pos={0 0}, upload=109688]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Tn9Ta9jyvkhsbyeATvZWfadDxVGHBU5PyE3fD3nVWh2Y9Kaf7): protocol: expected piece hash; context canceled; EOF Aug 13 02:48:08.666745024 (105h22m3.097825165s) [seg_pos={0 0}, upload=109688]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:48:08.666829095 (105h22m3.097909246s) [upload=109688]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:48:08.666834735 (105h22m3.097914886s) [upload=109688]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:48:08.979301278 (105h22m3.410381399s) [upload=109688]: Done uploading segments... Aug 13 02:48:08.979320665 (105h22m3.410400796s) [upload=109688]: Upload finished. err: <nil> Duration: 4.189971672s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [X...............X.....................X.......X.............] duration: 1.844708339s FAIL (2025-08-13 02:47:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.844732243s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................X.........................X..] duration: 5.845576208s FAIL (2025-08-13 02:47:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 02:46:28.515256865 (105h20m22.946336996s) [upload=109663]: Starting upload Aug 13 02:46:28.515314195 (105h20m22.946394316s) [upload=109663]: Done starting upload Aug 13 02:46:28.515421019 (105h20m22.94650115s) [upload=109663]: Uploading segments... Aug 13 02:46:28.515486996 (105h20m22.946567127s) [upload=109663]: Got next segment. Inline: false Aug 13 02:46:28.515519037 (105h20m22.946599168s) [upload=109663]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:46:28.515523305 (105h20m22.946603426s) [upload=109663]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:46:28.971104866 (105h20m23.402184997s) [seg_pos={0 0}, upload=109663]: Begin upload segment... Aug 13 02:46:28.971143912 (105h20m23.402224033s) [seg_pos={0 0}, upload=109663]: Done begin upload segment. Aug 13 02:46:28.971167026 (105h20m23.402247157s) [noise=true , address=104.36.82.38:4689 , piece_id=NOHZTFP3RC6SJ54PGSTU63K66IKRLOXI2AN62HPRYV5WHNO3LYBQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.971575397 (105h20m23.402655529s) [noise=true , address=104.36.82.58:4291 , piece_id=6SVJHUA2GFEWDQSH6YE4Q7RAKU4BABGIBDSWIYDNKYZDDCSEP43Q, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.971906391 (105h20m23.402986522s) [noise=true , address=104.36.82.35:4602 , piece_id=7VKAZ2K6DSWVEZAHQXF5RCSAMEJ7YSY4IRXYQHITW5I7P7LKIFYA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.971942600 (105h20m23.403022731s) [noise=true , address=104.36.82.42:4797 , piece_id=P6T4VQXHP4WFYHYLKME5F3MEFVP3OEJ3ESRKEWPZCZD4P5QII5RA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.972321976 (105h20m23.403402107s) [noise=true , address=104.36.82.9:4020 , piece_id=7S4CNG4XNCAPSGK7I2JU5JSQTBED5GTV7WNC5OM3HDMEISEYO4RA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.972360239 (105h20m23.40344037s) [noise=true , address=104.36.82.44:4872 , piece_id=JVEL4W57M5IBYKWGNQN2YW2PIWZJ2ADHVH3VZ2OIK6N722ECHFKQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.972668910 (105h20m23.403749031s) [noise=true , address=104.36.82.51:4051 , piece_id=6TGEF2U2BEKWGHZY74KUVROZZV35E4WXSPWL72NO7USW5QHAQOPQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.972693496 (105h20m23.403773617s) [noise=true , address=104.36.82.59:4314 , piece_id=REZLOYP3MP6WNGEL27KIX3F7FLACOOMOPRYF6HWNKH46ITOSO7UQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.972902566 (105h20m23.403982698s) [noise=true , address=64.49.13.140:4505 , piece_id=TK64CY6E2KKT5TGR2D6CVRFBSEEWWUHBCI67ZFCRGKQNKMLCZXMQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.973043636 (105h20m23.404123767s) [noise=true , address=104.36.82.3:4780 , piece_id=NWCGEYOOUKRWXRMBJPRBU3QPQOXYHVANJG5MQMP2OBA7LKPFSLTA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.973455033 (105h20m23.404535164s) [noise=true , address=104.36.82.37:4663 , piece_id=J2HQDTAJLLP3M22MOWPWU7VZNY4VDWLT2KD4WP5TKFDTF344IAIQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.973825904 (105h20m23.404906035s) [noise=true , address=104.36.82.60:4358 , piece_id=7GFNW4YK5WT7C3DTR7KKE7ZGN2WPZ75RMA7WQKG72RC45Z6ESFTQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.974373882 (105h20m23.405454003s) [noise=true , address=104.36.82.36:4650 , piece_id=K3NBHWRU5BKQOOJIKRVHYSQ24XI352KUUWTNQEI5YFMFM5TMBC4Q, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.974644730 (105h20m23.405724861s) [noise=true , address=104.36.82.34:4549 , piece_id=IPRI2SR3UPJZVEV6WEZYJ6XHIA2ZGKPRGD4AD74VMVGO4UMJVGMA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.974886262 (105h20m23.405966393s) [noise=true , address=79.127.201.215:4360 , piece_id=NHGGXO47YPMCNJN6P6SH6TBX57WNVH7ZLNJGMO2DJZVAKFO63MBA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.975129147 (105h20m23.406209268s) [noise=true , address=104.36.82.18:4382 , piece_id=UPEYVWHCVPIAMT2NU2GNLBUYYXI7Z6WOTEBBVCGZFEZMSFWZ7HZQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.975369467 (105h20m23.406449588s) [noise=true , address=77.90.31.13:4637 , piece_id=2N4XQ54VZGUU5E62HYQXZ6TB4XYLI73BITAD5NUTBSYVOHC5ES4Q, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.975606070 (105h20m23.406686201s) [noise=true , address=104.36.82.46:4482 , piece_id=HF5ZD4MO5BVKE5ZXADEDSDYVSHIEMV5QJBJHHSZFSUT2627XWDKQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:30.114349913 (105h20m24.545430044s) [upload=109663]: Next returned nil segment Aug 13 02:46:30.114368809 (105h20m24.54544894s) [upload=109663]: Waiting for error group managing segments... Aug 13 02:46:30.699874778 (105h20m25.130954909s) [noise=true , address=64.49.13.140:4505 , piece_id=TK64CY6E2KKT5TGR2D6CVRFBSEEWWUHBCI67ZFCRGKQNKMLCZXMQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.923090524 (105h20m25.354170655s) [noise=true , address=104.36.82.38:4689 , piece_id=NOHZTFP3RC6SJ54PGSTU63K66IKRLOXI2AN62HPRYV5WHNO3LYBQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.924630270 (105h20m25.355710401s) [noise=true , address=104.36.82.36:4650 , piece_id=K3NBHWRU5BKQOOJIKRVHYSQ24XI352KUUWTNQEI5YFMFM5TMBC4Q, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.941634796 (105h20m25.372714927s) [noise=true , address=104.36.82.42:4797 , piece_id=P6T4VQXHP4WFYHYLKME5F3MEFVP3OEJ3ESRKEWPZCZD4P5QII5RA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.941874695 (105h20m25.372954826s) [noise=true , address=77.90.31.13:4637 , piece_id=2N4XQ54VZGUU5E62HYQXZ6TB4XYLI73BITAD5NUTBSYVOHC5ES4Q, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.943655522 (105h20m25.374735653s) [noise=true , address=104.36.82.44:4872 , piece_id=JVEL4W57M5IBYKWGNQN2YW2PIWZJ2ADHVH3VZ2OIK6N722ECHFKQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.947885535 (105h20m25.378965666s) [noise=true , address=104.36.82.59:4314 , piece_id=REZLOYP3MP6WNGEL27KIX3F7FLACOOMOPRYF6HWNKH46ITOSO7UQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.950000281 (105h20m25.381080412s) [noise=true , address=104.36.82.58:4291 , piece_id=6SVJHUA2GFEWDQSH6YE4Q7RAKU4BABGIBDSWIYDNKYZDDCSEP43Q, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.974905273 (105h20m25.405985404s) [noise=true , address=104.36.82.3:4780 , piece_id=NWCGEYOOUKRWXRMBJPRBU3QPQOXYHVANJG5MQMP2OBA7LKPFSLTA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.998735589 (105h20m25.42981572s) [noise=true , address=104.36.82.60:4358 , piece_id=7GFNW4YK5WT7C3DTR7KKE7ZGN2WPZ75RMA7WQKG72RC45Z6ESFTQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:31.031579339 (105h20m25.46265947s) [noise=true , address=104.36.82.37:4663 , piece_id=J2HQDTAJLLP3M22MOWPWU7VZNY4VDWLT2KD4WP5TKFDTF344IAIQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:31.057431975 (105h20m25.488512116s) [noise=true , address=104.36.82.35:4602 , piece_id=7VKAZ2K6DSWVEZAHQXF5RCSAMEJ7YSY4IRXYQHITW5I7P7LKIFYA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:31.166499958 (105h20m25.597580089s) [noise=true , address=104.36.82.34:4549 , piece_id=IPRI2SR3UPJZVEV6WEZYJ6XHIA2ZGKPRGD4AD74VMVGO4UMJVGMA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:31.177127703 (105h20m25.608207834s) [noise=true , address=104.36.82.46:4482 , piece_id=HF5ZD4MO5BVKE5ZXADEDSDYVSHIEMV5QJBJHHSZFSUT2627XWDKQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:31.177160816 (105h20m25.608240967s) [seg_pos={0 0}, upload=109663]: Segment reached optimal threshold of 14 pieces. Aug 13 02:46:31.177343636 (105h20m25.608423767s) [noise=true , address=104.36.82.18:4382 , piece_id=UPEYVWHCVPIAMT2NU2GNLBUYYXI7Z6WOTEBBVCGZFEZMSFWZ7HZQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S3SoJPMMyHpqDgG5Ry1KErPGHXugcTpikGNPjpwC7qq5sH9ho): protocol: expected piece hash; context canceled; EOF Aug 13 02:46:31.177433908 (105h20m25.608514049s) [noise=true , address=79.127.201.215:4360 , piece_id=NHGGXO47YPMCNJN6P6SH6TBX57WNVH7ZLNJGMO2DJZVAKFO63MBA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 02:46:31.177496247 (105h20m25.608576378s) [noise=true , address=104.36.82.51:4051 , piece_id=6TGEF2U2BEKWGHZY74KUVROZZV35E4WXSPWL72NO7USW5QHAQOPQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n): protocol: expected piece hash; context canceled; EOF Aug 13 02:46:31.177522227 (105h20m25.608602358s) [noise=true , address=104.36.82.9:4020 , piece_id=7S4CNG4XNCAPSGK7I2JU5JSQTBED5GTV7WNC5OM3HDMEISEYO4RA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXPyMaA6o5DFhMTkkWf14bdqZoZ86MNjxzfSqvNU2KQj8xSyv): protocol: write tcp 188.166.225.170:38190->104.36.82.9:4020: use of closed network connection; write tcp 188.166.225.170:38190->104.36.82.9:4020: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38190->104.36.82.9:4020: use of closed network connection Aug 13 02:46:31.177546043 (105h20m25.608626174s) [seg_pos={0 0}, upload=109663]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:46:31.177629943 (105h20m25.608710074s) [upload=109663]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:46:31.177634241 (105h20m25.608714362s) [upload=109663]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:46:31.606955809 (105h20m26.03803594s) [upload=109663]: Done uploading segments... Aug 13 02:46:31.606973042 (105h20m26.038053173s) [upload=109663]: Upload finished. err: <nil> Duration: 5.845602409s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.X...............X.....................X.......X............] duration: 1.849625436s FAIL (2025-08-13 02:47:25 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.849651276s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X............X...................................] duration: 7.514481387s FAIL (2025-08-13 02:47:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.514502798s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................X.........................X.] duration: 5.822266486s FAIL (2025-08-13 02:46:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 02:46:28.515256865 (105h20m22.946336996s) [upload=109663]: Starting upload Aug 13 02:46:28.515314195 (105h20m22.946394316s) [upload=109663]: Done starting upload Aug 13 02:46:28.515421019 (105h20m22.94650115s) [upload=109663]: Uploading segments... Aug 13 02:46:28.515486996 (105h20m22.946567127s) [upload=109663]: Got next segment. Inline: false Aug 13 02:46:28.515519037 (105h20m22.946599168s) [upload=109663]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:46:28.515523305 (105h20m22.946603426s) [upload=109663]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:46:28.971104866 (105h20m23.402184997s) [seg_pos={0 0}, upload=109663]: Begin upload segment... Aug 13 02:46:28.971143912 (105h20m23.402224033s) [seg_pos={0 0}, upload=109663]: Done begin upload segment. Aug 13 02:46:28.971167026 (105h20m23.402247157s) [noise=true , address=104.36.82.38:4689 , piece_id=NOHZTFP3RC6SJ54PGSTU63K66IKRLOXI2AN62HPRYV5WHNO3LYBQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.971575397 (105h20m23.402655529s) [noise=true , address=104.36.82.58:4291 , piece_id=6SVJHUA2GFEWDQSH6YE4Q7RAKU4BABGIBDSWIYDNKYZDDCSEP43Q, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.971906391 (105h20m23.402986522s) [noise=true , address=104.36.82.35:4602 , piece_id=7VKAZ2K6DSWVEZAHQXF5RCSAMEJ7YSY4IRXYQHITW5I7P7LKIFYA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.971942600 (105h20m23.403022731s) [noise=true , address=104.36.82.42:4797 , piece_id=P6T4VQXHP4WFYHYLKME5F3MEFVP3OEJ3ESRKEWPZCZD4P5QII5RA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.972321976 (105h20m23.403402107s) [noise=true , address=104.36.82.9:4020 , piece_id=7S4CNG4XNCAPSGK7I2JU5JSQTBED5GTV7WNC5OM3HDMEISEYO4RA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.972360239 (105h20m23.40344037s) [noise=true , address=104.36.82.44:4872 , piece_id=JVEL4W57M5IBYKWGNQN2YW2PIWZJ2ADHVH3VZ2OIK6N722ECHFKQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.972668910 (105h20m23.403749031s) [noise=true , address=104.36.82.51:4051 , piece_id=6TGEF2U2BEKWGHZY74KUVROZZV35E4WXSPWL72NO7USW5QHAQOPQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.972693496 (105h20m23.403773617s) [noise=true , address=104.36.82.59:4314 , piece_id=REZLOYP3MP6WNGEL27KIX3F7FLACOOMOPRYF6HWNKH46ITOSO7UQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.972902566 (105h20m23.403982698s) [noise=true , address=64.49.13.140:4505 , piece_id=TK64CY6E2KKT5TGR2D6CVRFBSEEWWUHBCI67ZFCRGKQNKMLCZXMQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.973043636 (105h20m23.404123767s) [noise=true , address=104.36.82.3:4780 , piece_id=NWCGEYOOUKRWXRMBJPRBU3QPQOXYHVANJG5MQMP2OBA7LKPFSLTA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.973455033 (105h20m23.404535164s) [noise=true , address=104.36.82.37:4663 , piece_id=J2HQDTAJLLP3M22MOWPWU7VZNY4VDWLT2KD4WP5TKFDTF344IAIQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.973825904 (105h20m23.404906035s) [noise=true , address=104.36.82.60:4358 , piece_id=7GFNW4YK5WT7C3DTR7KKE7ZGN2WPZ75RMA7WQKG72RC45Z6ESFTQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.974373882 (105h20m23.405454003s) [noise=true , address=104.36.82.36:4650 , piece_id=K3NBHWRU5BKQOOJIKRVHYSQ24XI352KUUWTNQEI5YFMFM5TMBC4Q, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.974644730 (105h20m23.405724861s) [noise=true , address=104.36.82.34:4549 , piece_id=IPRI2SR3UPJZVEV6WEZYJ6XHIA2ZGKPRGD4AD74VMVGO4UMJVGMA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.974886262 (105h20m23.405966393s) [noise=true , address=79.127.201.215:4360 , piece_id=NHGGXO47YPMCNJN6P6SH6TBX57WNVH7ZLNJGMO2DJZVAKFO63MBA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.975129147 (105h20m23.406209268s) [noise=true , address=104.36.82.18:4382 , piece_id=UPEYVWHCVPIAMT2NU2GNLBUYYXI7Z6WOTEBBVCGZFEZMSFWZ7HZQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.975369467 (105h20m23.406449588s) [noise=true , address=77.90.31.13:4637 , piece_id=2N4XQ54VZGUU5E62HYQXZ6TB4XYLI73BITAD5NUTBSYVOHC5ES4Q, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.975606070 (105h20m23.406686201s) [noise=true , address=104.36.82.46:4482 , piece_id=HF5ZD4MO5BVKE5ZXADEDSDYVSHIEMV5QJBJHHSZFSUT2627XWDKQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:30.114349913 (105h20m24.545430044s) [upload=109663]: Next returned nil segment Aug 13 02:46:30.114368809 (105h20m24.54544894s) [upload=109663]: Waiting for error group managing segments... Aug 13 02:46:30.699874778 (105h20m25.130954909s) [noise=true , address=64.49.13.140:4505 , piece_id=TK64CY6E2KKT5TGR2D6CVRFBSEEWWUHBCI67ZFCRGKQNKMLCZXMQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.923090524 (105h20m25.354170655s) [noise=true , address=104.36.82.38:4689 , piece_id=NOHZTFP3RC6SJ54PGSTU63K66IKRLOXI2AN62HPRYV5WHNO3LYBQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.924630270 (105h20m25.355710401s) [noise=true , address=104.36.82.36:4650 , piece_id=K3NBHWRU5BKQOOJIKRVHYSQ24XI352KUUWTNQEI5YFMFM5TMBC4Q, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.941634796 (105h20m25.372714927s) [noise=true , address=104.36.82.42:4797 , piece_id=P6T4VQXHP4WFYHYLKME5F3MEFVP3OEJ3ESRKEWPZCZD4P5QII5RA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.941874695 (105h20m25.372954826s) [noise=true , address=77.90.31.13:4637 , piece_id=2N4XQ54VZGUU5E62HYQXZ6TB4XYLI73BITAD5NUTBSYVOHC5ES4Q, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.943655522 (105h20m25.374735653s) [noise=true , address=104.36.82.44:4872 , piece_id=JVEL4W57M5IBYKWGNQN2YW2PIWZJ2ADHVH3VZ2OIK6N722ECHFKQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.947885535 (105h20m25.378965666s) [noise=true , address=104.36.82.59:4314 , piece_id=REZLOYP3MP6WNGEL27KIX3F7FLACOOMOPRYF6HWNKH46ITOSO7UQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.950000281 (105h20m25.381080412s) [noise=true , address=104.36.82.58:4291 , piece_id=6SVJHUA2GFEWDQSH6YE4Q7RAKU4BABGIBDSWIYDNKYZDDCSEP43Q, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.974905273 (105h20m25.405985404s) [noise=true , address=104.36.82.3:4780 , piece_id=NWCGEYOOUKRWXRMBJPRBU3QPQOXYHVANJG5MQMP2OBA7LKPFSLTA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.998735589 (105h20m25.42981572s) [noise=true , address=104.36.82.60:4358 , piece_id=7GFNW4YK5WT7C3DTR7KKE7ZGN2WPZ75RMA7WQKG72RC45Z6ESFTQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:31.031579339 (105h20m25.46265947s) [noise=true , address=104.36.82.37:4663 , piece_id=J2HQDTAJLLP3M22MOWPWU7VZNY4VDWLT2KD4WP5TKFDTF344IAIQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:31.057431975 (105h20m25.488512116s) [noise=true , address=104.36.82.35:4602 , piece_id=7VKAZ2K6DSWVEZAHQXF5RCSAMEJ7YSY4IRXYQHITW5I7P7LKIFYA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:31.166499958 (105h20m25.597580089s) [noise=true , address=104.36.82.34:4549 , piece_id=IPRI2SR3UPJZVEV6WEZYJ6XHIA2ZGKPRGD4AD74VMVGO4UMJVGMA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:31.177127703 (105h20m25.608207834s) [noise=true , address=104.36.82.46:4482 , piece_id=HF5ZD4MO5BVKE5ZXADEDSDYVSHIEMV5QJBJHHSZFSUT2627XWDKQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:31.177160816 (105h20m25.608240967s) [seg_pos={0 0}, upload=109663]: Segment reached optimal threshold of 14 pieces. Aug 13 02:46:31.177343636 (105h20m25.608423767s) [noise=true , address=104.36.82.18:4382 , piece_id=UPEYVWHCVPIAMT2NU2GNLBUYYXI7Z6WOTEBBVCGZFEZMSFWZ7HZQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S3SoJPMMyHpqDgG5Ry1KErPGHXugcTpikGNPjpwC7qq5sH9ho): protocol: expected piece hash; context canceled; EOF Aug 13 02:46:31.177433908 (105h20m25.608514049s) [noise=true , address=79.127.201.215:4360 , piece_id=NHGGXO47YPMCNJN6P6SH6TBX57WNVH7ZLNJGMO2DJZVAKFO63MBA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 02:46:31.177496247 (105h20m25.608576378s) [noise=true , address=104.36.82.51:4051 , piece_id=6TGEF2U2BEKWGHZY74KUVROZZV35E4WXSPWL72NO7USW5QHAQOPQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n): protocol: expected piece hash; context canceled; EOF Aug 13 02:46:31.177522227 (105h20m25.608602358s) [noise=true , address=104.36.82.9:4020 , piece_id=7S4CNG4XNCAPSGK7I2JU5JSQTBED5GTV7WNC5OM3HDMEISEYO4RA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXPyMaA6o5DFhMTkkWf14bdqZoZ86MNjxzfSqvNU2KQj8xSyv): protocol: write tcp 188.166.225.170:38190->104.36.82.9:4020: use of closed network connection; write tcp 188.166.225.170:38190->104.36.82.9:4020: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38190->104.36.82.9:4020: use of closed network connection Aug 13 02:46:31.177546043 (105h20m25.608626174s) [seg_pos={0 0}, upload=109663]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:46:31.177629943 (105h20m25.608710074s) [upload=109663]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:46:31.177634241 (105h20m25.608714362s) [upload=109663]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:46:31.606955809 (105h20m26.03803594s) [upload=109663]: Done uploading segments... Aug 13 02:46:31.606973042 (105h20m26.038053173s) [upload=109663]: Upload finished. err: <nil> Duration: 5.82229428s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X............X..................................] duration: 7.037945156s FAIL (2025-08-13 02:46:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.037968129s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X.........................X] duration: 4.188931479s FAIL (2025-08-13 02:46:32 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 13 02:46:28.515256865 (105h20m22.946336996s) [upload=109663]: Starting upload Aug 13 02:46:28.515314195 (105h20m22.946394316s) [upload=109663]: Done starting upload Aug 13 02:46:28.515421019 (105h20m22.94650115s) [upload=109663]: Uploading segments... Aug 13 02:46:28.515486996 (105h20m22.946567127s) [upload=109663]: Got next segment. Inline: false Aug 13 02:46:28.515519037 (105h20m22.946599168s) [upload=109663]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:46:28.515523305 (105h20m22.946603426s) [upload=109663]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:46:28.971104866 (105h20m23.402184997s) [seg_pos={0 0}, upload=109663]: Begin upload segment... Aug 13 02:46:28.971143912 (105h20m23.402224033s) [seg_pos={0 0}, upload=109663]: Done begin upload segment. Aug 13 02:46:28.971167026 (105h20m23.402247157s) [noise=true , address=104.36.82.38:4689 , piece_id=NOHZTFP3RC6SJ54PGSTU63K66IKRLOXI2AN62HPRYV5WHNO3LYBQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.971575397 (105h20m23.402655529s) [noise=true , address=104.36.82.58:4291 , piece_id=6SVJHUA2GFEWDQSH6YE4Q7RAKU4BABGIBDSWIYDNKYZDDCSEP43Q, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.971906391 (105h20m23.402986522s) [noise=true , address=104.36.82.35:4602 , piece_id=7VKAZ2K6DSWVEZAHQXF5RCSAMEJ7YSY4IRXYQHITW5I7P7LKIFYA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.971942600 (105h20m23.403022731s) [noise=true , address=104.36.82.42:4797 , piece_id=P6T4VQXHP4WFYHYLKME5F3MEFVP3OEJ3ESRKEWPZCZD4P5QII5RA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.972321976 (105h20m23.403402107s) [noise=true , address=104.36.82.9:4020 , piece_id=7S4CNG4XNCAPSGK7I2JU5JSQTBED5GTV7WNC5OM3HDMEISEYO4RA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.972360239 (105h20m23.40344037s) [noise=true , address=104.36.82.44:4872 , piece_id=JVEL4W57M5IBYKWGNQN2YW2PIWZJ2ADHVH3VZ2OIK6N722ECHFKQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.972668910 (105h20m23.403749031s) [noise=true , address=104.36.82.51:4051 , piece_id=6TGEF2U2BEKWGHZY74KUVROZZV35E4WXSPWL72NO7USW5QHAQOPQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.972693496 (105h20m23.403773617s) [noise=true , address=104.36.82.59:4314 , piece_id=REZLOYP3MP6WNGEL27KIX3F7FLACOOMOPRYF6HWNKH46ITOSO7UQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.972902566 (105h20m23.403982698s) [noise=true , address=64.49.13.140:4505 , piece_id=TK64CY6E2KKT5TGR2D6CVRFBSEEWWUHBCI67ZFCRGKQNKMLCZXMQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.973043636 (105h20m23.404123767s) [noise=true , address=104.36.82.3:4780 , piece_id=NWCGEYOOUKRWXRMBJPRBU3QPQOXYHVANJG5MQMP2OBA7LKPFSLTA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.973455033 (105h20m23.404535164s) [noise=true , address=104.36.82.37:4663 , piece_id=J2HQDTAJLLP3M22MOWPWU7VZNY4VDWLT2KD4WP5TKFDTF344IAIQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.973825904 (105h20m23.404906035s) [noise=true , address=104.36.82.60:4358 , piece_id=7GFNW4YK5WT7C3DTR7KKE7ZGN2WPZ75RMA7WQKG72RC45Z6ESFTQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.974373882 (105h20m23.405454003s) [noise=true , address=104.36.82.36:4650 , piece_id=K3NBHWRU5BKQOOJIKRVHYSQ24XI352KUUWTNQEI5YFMFM5TMBC4Q, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.974644730 (105h20m23.405724861s) [noise=true , address=104.36.82.34:4549 , piece_id=IPRI2SR3UPJZVEV6WEZYJ6XHIA2ZGKPRGD4AD74VMVGO4UMJVGMA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.974886262 (105h20m23.405966393s) [noise=true , address=79.127.201.215:4360 , piece_id=NHGGXO47YPMCNJN6P6SH6TBX57WNVH7ZLNJGMO2DJZVAKFO63MBA, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.975129147 (105h20m23.406209268s) [noise=true , address=104.36.82.18:4382 , piece_id=UPEYVWHCVPIAMT2NU2GNLBUYYXI7Z6WOTEBBVCGZFEZMSFWZ7HZQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.975369467 (105h20m23.406449588s) [noise=true , address=77.90.31.13:4637 , piece_id=2N4XQ54VZGUU5E62HYQXZ6TB4XYLI73BITAD5NUTBSYVOHC5ES4Q, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:28.975606070 (105h20m23.406686201s) [noise=true , address=104.36.82.46:4482 , piece_id=HF5ZD4MO5BVKE5ZXADEDSDYVSHIEMV5QJBJHHSZFSUT2627XWDKQ, seg_pos={0 0}, upload=109663]: Uploading piece... Aug 13 02:46:30.114349913 (105h20m24.545430044s) [upload=109663]: Next returned nil segment Aug 13 02:46:30.114368809 (105h20m24.54544894s) [upload=109663]: Waiting for error group managing segments... Aug 13 02:46:30.699874778 (105h20m25.130954909s) [noise=true , address=64.49.13.140:4505 , piece_id=TK64CY6E2KKT5TGR2D6CVRFBSEEWWUHBCI67ZFCRGKQNKMLCZXMQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.923090524 (105h20m25.354170655s) [noise=true , address=104.36.82.38:4689 , piece_id=NOHZTFP3RC6SJ54PGSTU63K66IKRLOXI2AN62HPRYV5WHNO3LYBQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.924630270 (105h20m25.355710401s) [noise=true , address=104.36.82.36:4650 , piece_id=K3NBHWRU5BKQOOJIKRVHYSQ24XI352KUUWTNQEI5YFMFM5TMBC4Q, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.941634796 (105h20m25.372714927s) [noise=true , address=104.36.82.42:4797 , piece_id=P6T4VQXHP4WFYHYLKME5F3MEFVP3OEJ3ESRKEWPZCZD4P5QII5RA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.941874695 (105h20m25.372954826s) [noise=true , address=77.90.31.13:4637 , piece_id=2N4XQ54VZGUU5E62HYQXZ6TB4XYLI73BITAD5NUTBSYVOHC5ES4Q, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.943655522 (105h20m25.374735653s) [noise=true , address=104.36.82.44:4872 , piece_id=JVEL4W57M5IBYKWGNQN2YW2PIWZJ2ADHVH3VZ2OIK6N722ECHFKQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.947885535 (105h20m25.378965666s) [noise=true , address=104.36.82.59:4314 , piece_id=REZLOYP3MP6WNGEL27KIX3F7FLACOOMOPRYF6HWNKH46ITOSO7UQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.950000281 (105h20m25.381080412s) [noise=true , address=104.36.82.58:4291 , piece_id=6SVJHUA2GFEWDQSH6YE4Q7RAKU4BABGIBDSWIYDNKYZDDCSEP43Q, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.974905273 (105h20m25.405985404s) [noise=true , address=104.36.82.3:4780 , piece_id=NWCGEYOOUKRWXRMBJPRBU3QPQOXYHVANJG5MQMP2OBA7LKPFSLTA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:30.998735589 (105h20m25.42981572s) [noise=true , address=104.36.82.60:4358 , piece_id=7GFNW4YK5WT7C3DTR7KKE7ZGN2WPZ75RMA7WQKG72RC45Z6ESFTQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:31.031579339 (105h20m25.46265947s) [noise=true , address=104.36.82.37:4663 , piece_id=J2HQDTAJLLP3M22MOWPWU7VZNY4VDWLT2KD4WP5TKFDTF344IAIQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:31.057431975 (105h20m25.488512116s) [noise=true , address=104.36.82.35:4602 , piece_id=7VKAZ2K6DSWVEZAHQXF5RCSAMEJ7YSY4IRXYQHITW5I7P7LKIFYA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:31.166499958 (105h20m25.597580089s) [noise=true , address=104.36.82.34:4549 , piece_id=IPRI2SR3UPJZVEV6WEZYJ6XHIA2ZGKPRGD4AD74VMVGO4UMJVGMA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:31.177127703 (105h20m25.608207834s) [noise=true , address=104.36.82.46:4482 , piece_id=HF5ZD4MO5BVKE5ZXADEDSDYVSHIEMV5QJBJHHSZFSUT2627XWDKQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: <nil> Aug 13 02:46:31.177160816 (105h20m25.608240967s) [seg_pos={0 0}, upload=109663]: Segment reached optimal threshold of 14 pieces. Aug 13 02:46:31.177343636 (105h20m25.608423767s) [noise=true , address=104.36.82.18:4382 , piece_id=UPEYVWHCVPIAMT2NU2GNLBUYYXI7Z6WOTEBBVCGZFEZMSFWZ7HZQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12S3SoJPMMyHpqDgG5Ry1KErPGHXugcTpikGNPjpwC7qq5sH9ho): protocol: expected piece hash; context canceled; EOF Aug 13 02:46:31.177433908 (105h20m25.608514049s) [noise=true , address=79.127.201.215:4360 , piece_id=NHGGXO47YPMCNJN6P6SH6TBX57WNVH7ZLNJGMO2DJZVAKFO63MBA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y7dgLXYLTGGFXzHQ5ZYJ4f4kWYCoyiBhGesd1xqbTbnYjxNzG): protocol: expected piece hash; context canceled; EOF Aug 13 02:46:31.177496247 (105h20m25.608576378s) [noise=true , address=104.36.82.51:4051 , piece_id=6TGEF2U2BEKWGHZY74KUVROZZV35E4WXSPWL72NO7USW5QHAQOPQ, seg_pos={0 0}, upload=109663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rPbTARUrq7YsD8X75ocnyEvC517itk79s3Dp1wUg72fVuBS8n): protocol: expected piece hash; context canceled; EOF Aug 13 02:46:31.177522227 (105h20m25.608602358s) [noise=true , address=104.36.82.9:4020 , piece_id=7S4CNG4XNCAPSGK7I2JU5JSQTBED5GTV7WNC5OM3HDMEISEYO4RA, seg_pos={0 0}, upload=109663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XXPyMaA6o5DFhMTkkWf14bdqZoZ86MNjxzfSqvNU2KQj8xSyv): protocol: write tcp 188.166.225.170:38190->104.36.82.9:4020: use of closed network connection; write tcp 188.166.225.170:38190->104.36.82.9:4020: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38190->104.36.82.9:4020: use of closed network connection Aug 13 02:46:31.177546043 (105h20m25.608626174s) [seg_pos={0 0}, upload=109663]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:46:31.177629943 (105h20m25.608710074s) [upload=109663]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:46:31.177634241 (105h20m25.608714362s) [upload=109663]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:46:31.606955809 (105h20m26.03803594s) [upload=109663]: Done uploading segments... Aug 13 02:46:31.606973042 (105h20m26.038053173s) [upload=109663]: Upload finished. err: <nil> Duration: 4.188956627s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..X...............X.....................X.......X...........] duration: 1.857699135s FAIL (2025-08-13 02:46:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.857724672s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X............X.................................] duration: 7.488776146s FAIL (2025-08-13 02:45:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.488803168s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...X...............X.....................X.......X..........] duration: 1.848053738s FAIL (2025-08-13 02:45:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.848083645s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [....X...............X.....................X.......X.........] duration: 1.901425366s FAIL (2025-08-13 02:45:27 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.901451857s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.695681475s FAIL (2025-08-13 02:45:25 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.695711151s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X............X................................] duration: 7.49614365s FAIL (2025-08-13 02:45:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.496169549s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X............X...............................] duration: 10.240234853s FAIL (2025-08-13 02:44:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.240256657s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.813304986s FAIL (2025-08-13 02:44:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.813331135s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.....X...............X.....................X.......X........] duration: 1.872800447s FAIL (2025-08-13 02:44:31 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.872828501s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X............X..............................] duration: 7.453198814s FAIL (2025-08-13 02:44:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.453224012s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X............X.............................] duration: 6.457066797s FAIL (2025-08-13 02:43:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.457433838s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.04705492s FAIL (2025-08-13 02:43:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 6.047080009s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [......X...............X.....................X.......X.......] duration: 1.890553955s FAIL (2025-08-13 02:43:31 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.890574364s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.804793481s FAIL (2025-08-13 02:43:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.804824611s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................X............X............................] duration: 7.050833694s FAIL (2025-08-13 02:43:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.050858101s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.599640217s FAIL (2025-08-13 02:42:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.599688509s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................X............X...........................] duration: 7.315198505s FAIL (2025-08-13 02:42:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.315225226s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.......X...............X.....................X.......X......] duration: 1.842680024s FAIL (2025-08-13 02:42:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.842710533s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.247274016s FAIL (2025-08-13 02:42:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 6.247321138s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................X............X..........................] duration: 7.465125042s FAIL (2025-08-13 02:42:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.465147134s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................X............X.........................] duration: 6.685112236s FAIL (2025-08-13 02:41:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.685134778s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.982307702s FAIL (2025-08-13 02:41:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.982361455s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [........X...............X.....................X.......X.....] duration: 1.852608612s FAIL (2025-08-13 02:41:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 1.852636775s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................X............X........................] duration: 7.457373545s FAIL (2025-08-13 02:41:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.457409584s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.863566375s FAIL (2025-08-13 02:41:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.863594527s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.........X...............X.....................X.......X....] duration: 2.006262163s FAIL (2025-08-13 02:40:32 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 2.00629196s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................X............X.......................] duration: 7.507567801s FAIL (2025-08-13 02:40:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.507633436s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.697639213s FAIL (2025-08-13 02:40:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== digest mismatch when downloading object (sj://uptime-checks-placement-20/orbiter_us_0/4M):: after 6c1fff6044cc2f39200954ed73ef69d162ff43cacee4b51ba8bc74b9df737057 != before a365532cb8beae790d9d32f8c85f6a24aab304a84bec01083d9d4da2744b8b8e ======LOGS====== Aug 13 02:26:05.441240950 (104h59m59.872321081s) [upload=109305]: Starting upload Aug 13 02:26:05.441291356 (104h59m59.872371477s) [upload=109305]: Done starting upload Aug 13 02:26:05.441391698 (104h59m59.872471829s) [upload=109305]: Uploading segments... Aug 13 02:26:05.441492081 (104h59m59.872572212s) [upload=109305]: Got next segment. Inline: false Aug 13 02:26:05.441522018 (104h59m59.872602149s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:26:05.441526927 (104h59m59.872607058s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 13 02:26:05.902497042 (105h0m0.333577173s) [seg_pos={0 0}, upload=109305]: Begin upload segment... Aug 13 02:26:05.902561315 (105h0m0.333641446s) [seg_pos={0 0}, upload=109305]: Done begin upload segment. Aug 13 02:26:05.902587525 (105h0m0.333667656s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903169849 (105h0m0.33424998s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903618658 (105h0m0.334698789s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.903845051 (105h0m0.334925182s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904012982 (105h0m0.335093114s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904187145 (105h0m0.335267276s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904394583 (105h0m0.335474714s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904472322 (105h0m0.335552453s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.904972338 (105h0m0.336052479s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905337607 (105h0m0.336417738s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905590991 (105h0m0.336671122s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.905892318 (105h0m0.336972449s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906122819 (105h0m0.337202951s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906344634 (105h0m0.337424755s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906558734 (105h0m0.337638855s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906777272 (105h0m0.337857403s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.906996531 (105h0m0.338076652s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:05.907207605 (105h0m0.338287736s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Uploading piece... Aug 13 02:26:07.059989038 (105h0m1.491069169s) [upload=109305]: Next returned nil segment Aug 13 02:26:07.060007584 (105h0m1.491087715s) [upload=109305]: Waiting for error group managing segments... Aug 13 02:26:07.395868174 (105h0m1.826948305s) [noise=true , address=66.23.193.175:4555 , piece_id=UOWF3KXNLNAWFFNNHXN7G3ZX2MQ4V5OXOMCRAZQAP4SPJCBYVVRA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.882365712 (105h0m2.313445863s) [noise=true , address=104.36.82.15:4261 , piece_id=47VNA7MQTIX6WPX77O27C6JHHVXOQBGNIMQ56XBFT5SZL2FQRQCQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.892287064 (105h0m2.323367195s) [noise=true , address=104.36.82.41:4777 , piece_id=PQEAUPGORNK4GTVZIVU67RQ7XDNEKLEZ5C75USHNLGOSTNCVQENA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.909132876 (105h0m2.340212997s) [noise=true , address=77.90.31.11:4583 , piece_id=ROC3QSQ3XWDFANJ6LIWA3IGREJYNQEB55H2YO4ONHVS464B5ZYAA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.919101850 (105h0m2.350181981s) [noise=true , address=104.36.82.8:4000 , piece_id=36AQU3MDJZ5VG6HVND44J3UMMINBAUCYBXOJAALYNBLYJEYOBDSA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:07.995393421 (105h0m2.426473552s) [noise=true , address=104.36.82.34:4563 , piece_id=X7JAVTTT77V662VVIAYNYKHMFGZT56DDJ22FPFMFP7FJOXD75KNA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.000280581 (105h0m2.431360722s) [noise=true , address=104.36.82.47:4542 , piece_id=2UM2RA6L2CXBWMR2A4PLWUWC4RLCCGOV2RGXQ7V5QRTRNRXWYSZQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.043785921 (105h0m2.474866052s) [noise=true , address=104.36.82.2:4756 , piece_id=H6G3SLKLORHOVIKWSIHDR4ANOIEPRFLAWWOHZGWEZIP6JM5YFJGA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.112689807 (105h0m2.543769938s) [noise=true , address=77.90.31.17:4783 , piece_id=GFHMWG2PGJPS2MOYONPPB4FESHFPQ4DSLAL2JDLMVVSBDRSKKPBA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.151243883 (105h0m2.582324004s) [noise=true , address=104.36.82.44:4897 , piece_id=RT5D3D3BGXWU7ZK32W6LSZK25MKC2QNQZPXEZ3E4W6G7MIL5Q7RQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.152011712 (105h0m2.583091833s) [noise=true , address=104.36.82.23:4450 , piece_id=I7PGE7WPXKYKXWLNZ5LMBXNRYLAXVR3AIEGHUY4F6YGOXESGS34Q, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.200380848 (105h0m2.631460979s) [noise=true , address=104.36.82.39:4733 , piece_id=KLNYJ7LAUJF7MF2BEJ7BHV45ZBQQU3CGCB2EAOSP65QTAF32SXPA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.269830608 (105h0m2.700910739s) [noise=true , address=104.36.82.36:4641 , piece_id=H3OIEAUMCAIUFXPGEY3DRF2IRZRVMLNRYKPV3QPL4L3VXFNYLICA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462687726 (105h0m2.893767857s) [noise=true , address=104.36.82.61:4382 , piece_id=WAYXTDLO5N2UP3AWFYCNNRW2DI7Y2TETIPF6E5KTUE5QWRE3B4MQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: <nil> Aug 13 02:26:08.462765415 (105h0m2.893845546s) [seg_pos={0 0}, upload=109305]: Segment reached optimal threshold of 14 pieces. Aug 13 02:26:08.462876227 (105h0m2.893956358s) [noise=true , address=104.36.82.44:4894 , piece_id=HUWVPXUKN5J7T5SKY3R5FYGPETSY4TZIROYW5CTXYSRMJ3B3XHUQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462928387 (105h0m2.894008518s) [noise=true , address=104.36.82.14:4226 , piece_id=YTKG4V5OOWZZ35AD4UBIW7FTW23IO6H7AS3SDYO3S2SNEOP2AAVQ, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126xWe3ELGmYQfSUk3tYtGTSa6LnGfDNWcGT164pR6tHEtFPAFt): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.462986758 (105h0m2.894066879s) [noise=true , address=104.36.82.36:4618 , piece_id=IRYEKGTS7WZYLNARV2JVLWJK4R2BPBID7QOWU53F5A5KB6KXVLIA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bUyv2E7SXv5BCBjxbjHp99e4dKkuEwZpNv9zbFqddhD11CTq8): protocol: expected piece hash; context canceled; EOF Aug 13 02:26:08.463026826 (105h0m2.894106957s) [noise=true , address=104.36.82.37:4659 , piece_id=HH5JNLWDDWZUNRRS4EJQNUHAEKJZ3RG6PLUFLDUP7AGB6GL5F5QA, seg_pos={0 0}, upload=109305]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UBuWLogtN8mv4nwXDpzGdQtbKDz1EfVENSjXC54QdFp2fJg99): protocol: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38208->104.36.82.37:4659: use of closed network connection Aug 13 02:26:08.463042265 (105h0m2.894122386s) [seg_pos={0 0}, upload=109305]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 13 02:26:08.463097580 (105h0m2.894177711s) [upload=109305]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 13 02:26:08.463101287 (105h0m2.894181418s) [upload=109305]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:26:08.767259762 (105h0m3.198339893s) [upload=109305]: Done uploading segments... Aug 13 02:26:08.767276563 (105h0m3.198356694s) [upload=109305]: Upload finished. err: <nil> Duration: 5.697673948s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................X............X......................] duration: 7.215124349s FAIL (2025-08-13 02:40:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.215140691s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..........X...............X.....................X.......X...] duration: 2.135170119s FAIL (2025-08-13 02:39:47 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 13 02:37:52.538013577 (105h11m46.969093708s) [upload=109513]: Starting upload Aug 13 02:37:52.538068462 (105h11m46.969148593s) [upload=109513]: Done starting upload Aug 13 02:37:52.538112627 (105h11m46.969192758s) [upload=109513]: Uploading segments... Aug 13 02:37:52.538157382 (105h11m46.969237513s) [upload=109513]: Got next segment. Inline: true Aug 13 02:37:52.538160889 (105h11m46.96924102s) [upload=109513]: Waiting for error group managing segments... Aug 13 02:37:52.538204242 (105h11m46.969284363s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 13 02:37:52.538208750 (105h11m46.969288871s) [upload=109513]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 13 02:37:52.538211435 (105h11m46.969291566s) [upload=109513]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 13 02:37:53.388999282 (105h11m47.820079413s) [upload=109513]: Done uploading segments... Aug 13 02:37:53.389019419 (105h11m47.82009955s) [upload=109513]: Upload finished. err: <nil> Duration:* 2.135195307s core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)